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Overview 



Although much is known about how to help welfare recipients find jobs, little is known about how to 
help them and other low-wage workers keep jobs or advance in the labor market. This report presents 
information on the effectiveness of a program in South Carolina that aimed to help former welfare re- 
cipients obtain jobs, work more steadily, and move up in the labor market. The program was run as part 
of the Employment Retention and Advancement (ERA) project, which is testing 15 programs across the 
country. The ERA project was conceived by the U.S. Department of Health and Human Services 
(HHS); it is being conducted by MDRC under contract to HHS, with additional funding from the U.S. 
Department of Labor (DOL). 

South Carolina's ERA program. Moving Up, which operated between September 2001 and April 2005 in six 
rural counties, attempted to contact and assist individuals who had left welfare for any reason between Octo- 
ber 1997 and December 2000. Typically, nonworking participants received help finding a job, and working 
participants received help staying in their job or moving up. The core of Moving Up was one-on-one case 
management, with staff aiming to provide or connect participants with a range of services, including job 
search assistance, short-term vocational training, and support services. The program also provided modest 
financial incentives to encourage and reward program engagement and employment achievements. 

Moving Up is being evaluated using a random assignment research design, whereby eligible individuals 
were assigned, through a lottery-like process, either to a program group, whose members were recruited 
for the ERA program, or to a control group, whose members were not recruited or eligible for ERA ser- 
vices but who could use other services in the community. The program’s effects were estimated by com- 
paring how the two groups fared over time. 



Key Findings 

• Engaging individuals in Moving Up was challenging. After extensive outreach efforts, staff lo- 
cated about three-fourths of the program group — most of whom had been off welfare for several 
years when they entered the study. Even then, staff had to market the program to individuals who 
were not required to take part in it, many of whom did not want or need services. Just under half of 
the program group were engaged in ERA services during the year after they entered the study, many 
of them not veiy intensively. Compared with results for the control group. Moving Up increased par- 
ticipation in some employment-related services, such as vocational training, but only modestly. 

• Overall, Moving Up had little effect on employment rates, earnings, employment retention, or 
advancement. During the year - after entering the study, members of the program and control groups 
had similar employment outcomes. Results for early enrol Ices in the study, whose follow-up data 
cover two years, suggest that program effects will not emerge during the second year after study en- 
try or later. Overall, Moving Up also did not affect welfare or food stamp receipt or income. The pro- 
gram, however, had positive effects on employment for three subgroups of sample members: those 
who had become unemployed shortly before entering the study, those who had left welfare less than 
two and a half years before entering the study, and those who had left welfare because of a sanction 
or the state’s time limit on benefit receipt. The effects for the recently unemployed subgroup are the 
largest but are less certain than the other results because the sample size for the group is small — 
only about 9 percent of the sample analyzed for this report, or 249 individuals. Finally, one county’s 
program produced positive effects on employment, but the other five did not. 

These results arc not the final word on South Carolina’s ERA program, as MDRC will continue to track 
employment outcomes for the study’s participants. The findings do, however, illustrate the persistent chal- 
lenge of encouraging participation in postemployment services and making a difference in labor market 
outcomes for welfare leavers. Many of the individuals in the study remain poor and in need of supports. 
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About the Employment Retention and 
Advancement Project 



The federal welfare overhaul of 1996 ushered in myriad policy changes aimed at getting 
low-income parents off public assistance and into employment. These changes — especially 
cash welfare’s transformation from an entitlement into a time-limited benefit contingent on 
work participation — have intensified the need to help low-income families become economi- 
cally self-sufficient and remain so in the long term. Although a fair amount is known about how 
to help welfare recipients prepare for and find jobs in the first place, the Employment Retention 
and Advancement (ERA) project is the most comprehensive effort thus far to discover which 
approaches help welfare recipients and other low-income people stay steadily employed and 
advance in their jobs. 

Launched in 1999 and slated to end in 2008, the ERA project encompasses more than a 
dozen demonstration programs and uses a rigorous research design to analyze the programs’ im- 
plementation and impacts on research sample members, who were randomly assigned to the study 
groups. With technical assistance from MDRC and The Lewin Group, the study was conceived 
and funded by the U.S. Department of Health and Human Services, Administration for Children 
and Families; supplemental support comes from the U.S. Department of Labor. Because the pro- 
grams’ aims and target populations vary, so do their services: 

• Advancement programs focus on helping low-income workers move into 
better jobs by offering such services as career counseling and education and 
training. 

• Placement and retention programs aim to help participants find and hold 
jobs and are aimed mostly at “hard-to-employ” people, such as welfare re- 
cipients who have disabilities or substance abuse problems. 

• Mixed-goals programs focus on job placement, retention, and advancement, 
in that order, and are targeted primarily to welfare recipients who are search- 
ing for jobs. 

The ERA project’s evaluation component investigates the following aspects of each 
program: 

• Implementation. What services does the program provide? How are those 
services delivered? Who receives them? How are problems addressed? 

• Impacts. To what extent does the program improve employment rates, job 
retention, advancement, and other key outcomes? How does it affect enrol- 
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lees’ children? Looking across programs, which approaches are most effec- 
tive, and for whom? 

A total of 15 ERA programs are being implemented in eight states: 

• California: Los Angeles County and Riverside County 

• Illinois: Cook County (Chicago) and St. Clair County (East St. Louis) 

• Minnesota: Hennepin County (Minneapolis) 

• New York: New York City 

• Ohio: Cleveland 

• Oregon: Eugene, Medford, Portland, and Salem 

• South Carolina: Pee Dee Region (six counties in the northeast comer of the 
state) 

• Texas: Corpus Christi, Fort Worth, and Houston 

The evaluation draws on administrative and fiscal records, surveys of participants, and 
field visits to the sites. 
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Executive Summary 



This report presents evidence on the implementation and effectiveness of a program in 
South Carolina that aimed to help former recipients of Temporary Assistance for Needy Fami- 
lies (TANF) obtain jobs, work steadily, and advance in the labor market. The program operated 
as part of the Employment Retention and Advancement (ERA) project, which is testing 15 pro- 
grams across the country. The ERA project was conceived and funded by the Administration 
for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) 
and is also supported by the U.S. Department of Fabor (DOF). The project is being conducted 
by MDRC, a nonprofit, nonpartisan research organization, under contract to HHS. 

Most of the results presented in this report are based on the year after individuals en- 
tered the study; a few are based on two years of follow-up. The results include the program’s 
effect on employment rates and stability, earnings, and advancement in the labor market. These 
interim results are important but are not the final word on the program, as MDRC will continue 
to track employment outcomes for the study’ s participants. 



The ERA Project 

Although much is known about effective strategies to help welfare recipients and other 
disadvantaged groups find jobs, little is known about how to help them and other low-wage 
workers keep jobs or advance in the labor market. Previously studied postemployment pro- 
grams were not found to improve participants’ outcomes. The ERA project was designed to 
build on past efforts and identify and test innovative programs designed to promote employ- 
ment stability and wage progression among welfare recipients or other low-income groups. 
From 2000 to 2003, a total of 15 ERA experiments were implemented in eight states, including 
South Carolina. 

The design of the evaluation is similar in most of the project’s sites. Individuals who 
meet the ERA eligibility criteria, which vary by site, are assigned, at random, to a program 
group, called the ERA group, or to a control group. Members of the ERA group are recruited 
for (and, in some sites, are required to participate in) the ERA program, while those in the con- 
trol group are not eligible for ERA services but can access other services and supports available 
in the community. MDRC is tracking both research groups over time. The random assignment 
process ensured that the two groups were comparable when they entered the study; thus, any 
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differences between them that emerge over time — for example, in employment rates or aver- 
age earnings — are attributable to the ERA program. 1 



South Carolina’s ERA Program 

South Carolina’s ERA program, called “Moving Up,” operated between September 2001 
and April 2005 and was developed by the state’ s Department of Social Services (DSS) in response 
to trends in the state’s welfare caseload and low-income working population. As in most states, in 
South Carolina, the welfare caseload decreased dramatically in the 1990s. This was, in part, a re- 
sult of the state’s short time limit on welfare — most families cannot receive TANF for more than 
2 years in a 10-year period — and a tough sanctioning policy in which a family’s grant can be 
closed if the parent does not comply with program requirements. In the late 1990s, South Carolina 
conducted research showing that, like welfare leavers across the country, some leavers in the state 
were not working; many were working but not steadily; and others were stuck in low-wage jobs. 
In an effort to help former recipients succeed in the labor market, state officials decided to reach 
out to them and offer support and services. They chose to target all welfare leavers, so the pro- 
gram was designed to provide services to those who were not working as well as to those who 
were working but could use help sustaining work or moving up. 

DSS chose to operate the ERA program in the Pee Dee Region, in the northeast part of 
South Carolina. This largely rural region encompasses six counties: Chesterfield, Darlington, 
Dillon, Florence, Marion, and Marlboro. The state chose this area because it is the most eco- 
nomically disadvantaged region in the state and because the DSS county directors there had ex- 
perience collaborating on prior efforts. 

The Moving Up program targeted people who had left the TANF rolls in the Pee Dee 
Region, for any reason, between October 1997 and December 2000 and who did not return to 
the rolls. Each month from September 2001 to January 2003, using the state’s TANF database, 
100 individuals were randomly selected from this eligible group to be in the site’s ERA group, 
and another 100 were randomly selected to serve as the study’s control group. Each of the ERA 
group members was assigned to one of 10 case managers in the counties, who then attempted to 
locate the individuals and engage them in the program. The control group members were not 
recruited or eligible for the ERA program, but they could participate in other programs available 
in the community. The sample analyzed for this report (the “report sample”) includes the 2,864 



'For more information on the ERA project, see Bloom, Anderson, Wavelet, Gardiner, and Fishman, New 
Strategies to Promote Stable Employment and Career Progression: An Introduction to the Employment Retention 
and Advancement Project (U.S. Department of Health and Human Services, 2002). For early results from four 
sites, including South Carolina, see Bloom, Hendra, Martinson, and Scrivener, The Employment Retention and 
Advancement Project: Early Results from Four Sites (U.S. Department of Health and Human Services, 2005). 
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individuals who were randomly assigned from September 2001 to December 2002. This repre- 
sents 94 percent of the site’s full research sample. 2 

The sample includes a diverse pool of TANF leavers. The length of time between the 
point that sample members left welfare and the point that they entered the study ranges from 
nine months to just over five years; almost three-fourths (72 percent) of the report sample had 
been off welfare for two and a half years or longer. They left for various reasons: 15 percent did 
so because they had reached the 24-month time limit; 19 percent had been sanctioned; and 40 
percent had begun to earn too much to qualify for benefits. The rest of the sample left for other 
reasons, including failing to provide necessary information for benefit redetermination. About 
half of the research sample members were working when they entered the study, and half were 
not. The vast majority are women, and nearly four out of five are African-American. 

Moving Up services varied depending on the participants’ needs, but the core of the 
program was one-on-one case management. Staff, called “career consultants,” worked with par- 
ticipants to understand their employment goals and develop an employment plan. Typically, 
participants who were not working received assistance preparing for and searching for a job, 
and those who were working received help staying in their job or moving up. Career consultants 
provided or connected participants with a range of services, including one-on-one job search 
assistance, job search classes, short-term vocational training, and support services, such as 
transportation assistance. The program provided modest financial incentives to encourage and 
reward program engagement and employment achievements. 

Program funding varied over time. When the study began, Moving Up was fully funded. 
Over time, South Carolina’s budget situation worsened, leading to funding cuts in many programs, 
including Moving Up. Career consultants remained on the job, but — for a period from late 2002 
through summer 2003 — most counties froze or limited spending on Moving Up’s financial incen- 
tives, education and training tuition payment, transportation assistance, and some other services. 



Key Findings on Program Implementation 

This section summarizes the report’s findings on how Moving Up was implemented 
and on sample members' participation in the program and other employment-related services. 
The findings are based on field research, a “time study” of career consultants, automated pro- 



sample members who entered the study in January 2003 are not included in this report because less than 
one year of earnings data were available for them when the analyses for this report were conducted. Some indi- 
viduals who had returned to the TANF rolls after December 2000 were erroneously selected for the sample; 
those individuals were dropped from both research groups and are not included in the analysis. 
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gram tracking data, and a survey administered to a subset of sample members about 12 months 
after they entered the study. Key implementation findings follow. 

• Locating and marketing Moving Up to potential participants was chal- 
lenging. During the year after they entered the study, just under half of 
the ERA group were engaged in program services, many of them not 
very intensively. 

The study’s target group included many individuals whose contact information in the 
state’s database was outdated, and thus it was time-consuming, if not impossible, to locate them. 
According to program records, after extensive outreach efforts, the program contacted, in person 
or by phone, about three-fourths of the ERA group within the year after they entered the study. 

Even then, staff still faced the challenge of marketing the program to individuals who were 
not required to take part in it and may not have wanted the program's assistance. Within a year of 
entering the study, just under half of the ERA group had been engaged in Moving Up. Some of 
these individuals had a lot of contact with the program and its services during that year (for example, 
they may have received a lot of help from a career consultant and participated in a job search class or 
vocational training), and some had more cursory contact (they may have had just a few contacts with 
a career consultant). During the year after entering the study, just under a third of the ERA group 
were engaged relatively intensively in Moving Up (they had at least four contacts with staff, at least 
two of which were in person, and they received at least one incentive payment). 

Because participation in Moving Up was voluntary, the only chance that the program had 
to affect individuals’ outcomes was by engaging them. A mandatory program, in contrast, can 
affect even nonparticipants, if they change their behavior in response to the mandate. The fact that 
just under half of the program group ever participated in the program and just under a third did so 
relatively intensively diminished the program’s ability to affect employment outcomes for the full 
research sample, since both participants and nonparticipants are included in the analysis. 

• Providing postemployment services was challenging. 

Delivery of retention and advancement services was strong in some of the Pee Dee 
counties but less so in others. DSS and most staff members had a lot of experience helping peo- 
ple prepare for and find jobs, but they had less experience serving employed clients. The site 
devoted considerable resources to staff development and training in these areas, but service de- 
livery remained challenging. 

Based on a time study that recorded career consultants’ activities over a two-week period, 
the most common activity in South Carolina during contact with working participants was “general 
check-in,” accounting for over a third of the contact. This proportion is higher than in most of the 
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other ERA sites, which suggests that, compared with most other programs in the study, Moving Up 
dealt less with specific issues regarding job placement, retention, and advancement. 

• Moving Up increased receipt of employment-related services, but only 
modestly. 

Based on data from the study’s survey, 44 percent of ERA group members had contact 
with a case manager or employment program during the year after entering the program, com- 
pared with 29 percent of the control group members. (It is not known specifically who control 
group members had contact with, but they were able to receive services from programs other 
than Moving Up and from other agencies in the community.) ERA group members were also 
somewhat more likely to have received retention and advancement services, to have participated 
in vocational training, and to have participated in education or training while employed. For 
example, 1 8 percent of the ERA group received help with retention and advancement — an in- 
crease of 10 percentage points above the control group’s mean of 8 percent. 

• Program implementation and participation varied by county. 

Although the program’s design was uniform across the six Pee Dee counties and the 
program coordinator encouraged consistent implementation, the program’s services and inten- 
sity varied somewhat. Only one of the counties substantially increased participation in all three 
strands of Moving Up’s services: employment retention, advancement, and placement (although 
the effect on placement just missed statistical significance). Based on this evidence and on in- 
formation from field research and the time study, it appears that this county, compared with the 
other five, operated a program that most closely approximates Moving Up’ s design. 



Key Findings on Program Impacts 

This section summarizes the effects that South Carolina’s ERA program had on sample 
members. The findings are based on administrative records data (earnings reported to both 
South Carolina’s and North Carolina’s unemployment insurance systems, along with TANF 
and food stamp payments from South Carolina) and data from the study’ s survey. The report' s 
key impact findings follow. 

• Moving Up had little effect on employment rates, earnings, employment 
retention, or advancement for the full research sample. 

The control group’ s experiences represent what would have happened in the absence of 
the ERA program. As Table ES.l shows, during the year following entry into the study, about 
two-thirds (68 percent) of the control group members were employed, and 40 percent worked 
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The Employment Retention and Advancement Project 
Table ES.l 

Summary of the ERA Program’s Impacts 
South Carolina 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Ever employed 3 (%) 


68.5 


67.8 


0.6 


0.64 


Average quarterly employment (%) 


55.1 


54.2 


0.9 


0.43 


Employed 4 consecutive quarters (%) 


40.2 


40.2 


0.1 


0.96 


Earnings ($) 


6,532 


6,743 


-211 


0.29 


Earned over $10,000 (%) 


28.1 


28.8 


-0.7 


0.58 


Ever received TANF (%) 


7.6 


7.2 


0.3 


0.74 


Amount of TANF received ($) 


62 


62 


0 


0.98 


Ever received food stamps (%) 


62.6 


61.9 


0.7 


0.58 


Amount of food stamps received ($) 


1.856 


1,904 


-49 


0.33 


Total measured income 13 ($) 


8.450 


8,710 


-260 


0.18 


Sample size (total = 2.864) 


1,421 


1,443 







SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of South 
Carolina and UI data from the State of North Carolina. 



NOTES: “This table includes only employment and earnings in jobs covered by the North Carolina and South 
Carolina unemployment insurance (UI) programs. It does not include employment outside North Carolina and 
South Carolina or in jobs not covered by UI (for example, "off the books” jobs, some agricultural jobs, and 
federal government jobs.) 

b This measure represents the sum of UI earnings, TANF, and food stamps. 

Estimates were regression-adjusted using ordinary least squares, controlling for pre -random assignment 
characteristics of sample members. 

Rounding may cause slight discrepancies in calculating sums and differences. 

A two-tailed t-test was applied to differences between outcomes for the program and control groups. 

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. 

Dollar averages include zero values for sample members who were not employed or were not receiving 
TANF or food stamps. 



in all four quarters. They earned an average of about $6,700. This average includes all control 
group members — both those who worked during the follow-up period and those who did not. 
Employed control group members earned an average of about $9,900 during the year (not shown 
in the table). Just over one-fourth (29 percent) of the control group earned more than $10,000. 
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Not surprisingly, given that South Carolina targeted TANF leavers — many who had 
been off the welfare rolls for some years — few in the control group received TANF benefits 
during the year after they entered the study. Roughly two-thirds, however, received food 
stamps. During the year, control group members received an average of about $8,700 from 
earnings, TANF, and food stamps. 

Administrative records provide only a partial view of sample members' available re- 
sources. To provide a more complete view, the study’s survey asked about all sources of in- 
come, including, for example, odd jobs, child support, Supplemental Security Income (SSI) 
benefits, and other household members’ earnings and other income. The control group reported 
that their household income in the month before they were interviewed was about $1,300, on 
average. This translates into an annual household income of $15,600. The average household 
for sample members in both research groups included four people, and the federal poverty rate 
for a family of four in 2003 was $18,400. (The survey outcomes are not shown in the table.) 3 

As Table ES.l shows, during the year after sample members entered the study, Moving 
Up did not increase employment rates or earnings. For example, during that year, 69 percent of 
the ERA group were employed, compared with 68 percent of the control group. Furthermore, 
the program did not affect employment retention or stability or advancement in the labor mar- 
ket: About the same proportion of sample members in each research group worked in all four 
quarters of the follow-up year and earned over $10,000 during the year. The program also did 
not affect the characteristics or quality of sample members' jobs (not shown in the table). Not 
surprisingly, the program did not affect TANF or food stamp receipt. It also did not increase 
income, whether measured using administrative records or the survey. 

Employment and earnings were also examined for an early cohort of sample members 
— randomly assigned from September to December 2001 — for whom an additional year of 
follow-up administrative records data were available. Among this cohort, the program increased 
employment rates in the last two quarters of the first year of follow-up. This may indicate that 
Moving Up was more effective earlier, when caseload sizes were smaller and the program was 
fully funded. The impacts, however, were short lived. Among the early cohort, the program did 
not affect employment or earnings in the second year of follow-up, and the trends in outcomes 
do not suggest that impacts will begin to emerge after the two-year period. 

Finally, a separate analysis identified the effects of Moving Up among those who were 
most likely to participate in the program. In order to conduct this analysis using experimental 
methods, a regression-based subgroup was created, defined using the baseline characteristics 

3 The survey did not ask about annual income, and it is unknown how typical that month's income was for 
sample members. The annual estimate is provided as a rough comparison with the annual figure based on the 
administrative records data. 
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that were most associated with eventual participation. This analysis found that Moving Up did 
not generate significant increases in employment and earnings even among those who were 
most likely to participate in the program. 

• South Carolina’s ERA program had positive effects for three subgroups 
of sample members: those who had become unemployed shortly before 
entering the study, recent TANE leavers, and those who had left TANF 
because of a sanction or the time limit. 

Findings for the full report sample may mask important results for different subgroups 
of individuals. In analyses for this report, various subgroups were defined using sample mem- 
bers' characteristics when they entered the study. Subgroups defined by education level, length 
of time receiving TANF, race/ethnicity, and whether the sample member received food stamps 
just before entering the study were examined, and no systematic differences were found. Three 
subgroup splits, however, yielded some interesting results. 

Moving Up produced gains for sample members who had become unemployed just be- 
fore entering the study. While these results are promising, they are less certain than the other 
subgroup results because the sample size is small: This subgroup comprises only 9 percent of 
the full sample, or 249 individuals. It includes sample members who did not work in the quarter 
before random assignment but who did work in at least two of the three quarters before that. In 
other words, they had employment experience but had recently become unemployed. Among 
this recently unemployed subgroup, Moving Up increased employment rates, retention, and ad- 
vancement. For example, the program increased the proportion of sample members who were 
employed by 13 percentage points, increased average annual earnings by about $1,800, and in- 
creased the proportion of sample members who earned over $10,000 by 1 1 percentage points. 

Program tracking data suggest that ERA group members in the recently unemployed 
subgroup were somewhat more likely to be engaged in Moving Up than other ERA group 
members. The survey data, however, do not suggest that the program increased participation, 
compared with control group levels, for this subgroup more than for others. The recently unem- 
ployed subgroup may have been better positioned than other sample members to benefit from 
Moving Up’s services. As noted earlier, program staff had more experience providing job 
placement services than retention and advancement services. Sample members who were em- 
ployed when they entered the study did not need placement help. Sample members who were 
unemployed but did not have recent work experience were harder to place in jobs and, thus, 
were less able to benefit from job placement services. The survey data also suggest that mem- 
bers of this longer-term unemployed (or mostly unemployed) subgroup were more likely to be 
in poor health and to live in a household with an employed adult. 
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The program also increased employment for sample members who had left TANF less 
than two and a half years before entering the study and those who had left because they were 
sanctioned or reached the time limit, but the effects for both groups were more limited than for 
the recently unemployed. The two subgroups make up, respectively, 28 percent and 35 percent 
of the full report sample. Higher proportions of ERA group members in these two subgroups 
were engaged in Moving Up, compared with other groups of sample members. The survey data, 
however, do not suggest that the program increased participation, compared with control group 
levels, for these subgroups more than for others. 

• Moving Up’s effects varied across the counties in the study. 

One county’ s ERA program — the one that stood out in the implementation and par- 
ticipation results — produced positive effects for sample members. The program increased the 
employment rate by 9 percentage points. None of the programs in the other five counties pro- 
duced positive effects on employment. 



Conclusions 

Moving Up is one of 15 programs being studied as part of the ERA project, and reports 
over the next two years will present results for the other programs. MDRC will continue to track 
sample members in South Carolina, using administrative records, and will make public longer- 
term results when they are available. (Although an early look at the findings two years after 
study entry are not promising, effects may emerge.) As the study continues to generate informa- 
tion, more definitive conclusions will be possible. However, some preliminary conclusions can 
be drawn based on the results in this report. 

• Implementing a retention and advancement program is challenging. 

Encouraging participation in postemployment services is difficult. Low-income, single, 
working parents are already juggling work, family, and other responsibilities, and it should not 
be surprising that many are reluctant to participate in job retention and advancement activities. 
South Carolina’s challenge was compounded by the fact that the target group was very broad 
and many potential participants were hard to locate. Also, offering postemployment services is 
new to most agencies and staff involved in the ERA project, including those in South Carolina. 
Despite considerable staff development and training, service delivery remained challenging. 

• It may have been especially difficult to implement South Carolina’s 
ERA model, which relied on individual case managers’ abilities to assess 
participants’ needs, skills, and goals and then to provide services that 
would make a difference. 
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Only one of the six Pee Dee counties succeeded in fully operating the Moving Up pro- 
gram as it was designed and in a way that improved individuals’ outcomes. The program relied 
heavily on one-on-one case management and only modestly increased participation in more 
concrete activities, such as vocational training. This approach may work better in a centralized, 
closely supervised setting than it did in this study. 

• For more positive results, a program like South Carolina’s could be tar- 
geted to those who want to participate and who are likely to benefit from 
the services. 

Moving Up targeted a very diverse group, many of whom were not interested in receiv- 
ing services and some of whom participated in services but were not helped. A program might 
achieve better results by advertising services to TANF leavers and serving those who come 
forward, rather than tracking down a wider, less enthusiastic group. 

• Many of the TANF leavers in the study remain poor and in need of 
supports. 

It is important to point out that the issue that prompted South Carolina to implement the 
Moving Up program remains salient. The group of TANF leavers in this study includes many 
who worked during the follow-up year but some who did not, and earnings, on average, remain 
relatively low. The outcome levels for both the control group and the ERA group highlight the 
importance of additional supports for low-income working families, as well as effective services 
to help them move up in the labor market. 
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Chapter 1 

Introduction 



To set the stage for the rest of the report, this chapter first provides an overview of the na- 
tional Employment Retention and Advancement (ERA) project, of which South Carolina’ s ERA 
program is a part. It then describes South Carolina’s ERA program, including the environment in 
which it was implemented and the program's target population. The chapter concludes by describ- 
ing the ERA evaluation in South Carolina and highlighting the contents of the remaining chapters. 



Overview of the National ERA Project 

For over a decade, policymakers and program operators have struggled to learn what 
kinds of services, supports, and incentives are best able to help low-income working parents retain 
steady employment and move up to better jobs. This issue has assumed even greater urgency in 
the wake of the 1990s welfare reforms, which made long-term welfare receipt much less feasible 
for families. Despite many efforts, scant evidence exists about effective strategies to promote em- 
ployment retention and advancement. Previously evaluated programs that were aimed at improv- 
ing retention or advancement — notably, the Post-Employment Services Demonstration (PESD), 
a four-site project that tested programs providing follow-up case management to welfare recipi- 
ents who found jobs — generally failed to improve employment outcomes. 

The Employment Retention and Advancement project was designed to improve on past 
efforts in this area by identifying and testing innovative models designed to promote employment 
stability and wage progression among welfare recipients and other low-income groups. The pro- 
ject began in 1998, when the U.S. Department of Health and Human Services (HHS) issued plan- 
ning grants to 13 states to develop new programs. The following year, HHS selected MDRC to 
conduct an evaluation of the ERA programs. 1 From 2000 to 2003, MDRC and its subcontractor, 
The Lewin Group, worked closely with the states that had received planning grants, and with sev- 
eral other states, to mount tests of ERA programs. MDRC, Lewin, and Cygnet Associates also 
provided extensive technical assistance to some of the states and program operators, since most 
were starting the project from scratch, with no proven models on which to build. 

Ultimately, a total of 15 ERA experiments were implemented in eight states, including 
South Carolina. Almost all the programs target current or former recipients of Temporary Assis- 
tance for Needy Families (TANF) — the cash welfare program that mainly serves single mothers 
and their children — but the program models are very diverse. One group of programs targets 



'The U.S. Department of Labor has also provided funding to support the ERA project. 
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low-wage workers and focuses on advancement. Another group targets individuals who are con- 
sidered “hard to employ” and primarily aims to place them in stable jobs. Finally, a third group of 
programs has mixed goals and targets a diverse set of populations, including former TANF recipi- 
ents, TANF applicants, and low-wage workers in particular firms. Some of these programs initiate 
services before individuals go to work, while others begin services after employment. Appendix 
Table A.l describes each of the ERA programs and identifies its goals and target populations. 

The evaluation design is similar in most of the sites. Individuals who meet ERA eligi- 
bility criteria (which vary from site to site) are assigned, at random, to the program group — 
also called the “ERA group” — or to the control group. Members of the ERA group are re- 
cruited for the ERA program (and, in some sites, are required to participate in it), whereas 
members of the control group are not eligible for ERA services. The extent and nature of the 
services and supports available to the control group vary from site to site. The random assign- 
ment process ensures that any differences in outcomes that emerge between the two research 
groups during the follow-up period can be confidently attributed to the ERA program, rather 
than to differences in the characteristics of the people in the groups. 



The South Carolina ERA Program: Moving Up 

Origins and Goals of the South Carolina ERA Program 

South Carolina’s ERA program, called “Moving Up,” operated in six largely rural 
counties in the Pee Dee Region, in the northeastern part of the state. This mixed-goal program 
operated from September 2001 through April 2005, and it provided both pre- and postemploy- 
ment services to former TANF recipients; the program included work placement, employment 
stabilization, and advancement services. 2 

Moving Up was developed in response to trends in the state’s TANF caseload and 
working-poor population. As in most states, South Carolina's TANF caseload decreased dra- 
matically in the 1990s. Between 1993 and 1998, for example, the number of TANF recipients 
dropped by more than half. The decrease resulted in part from the state’s aggressive welfare 
reform program, Family Independence. Instituted in 1995, this program imposes a short time 
limit on benefit receipt and includes tough penalties for noncompliance with program rules. 
Specifically, most of South Carolina’s TANF recipients are limited to no more than 24 months 
of assistance in a 10-year period and to no more than 60 months in their lifetime. 3 Recipients 
who do not meet the Family Independence work and training requirements can have their bene- 



2 The state operated a pilot program for about 250 participants from June through August 2001. 

3 South Carolina Department of Social Services, Office of Family Assistance, 2000-2001. 
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fits discontinued, which is called a “full-family sanction.” 4 Early in the ERA evaluation’s study 
period, South Carolina had one of the highest sanctioning rates in the country. 5 Later, sanction- 
ing rates dropped substantially, as the state began to use sanctions as a last resort. The cash grant 
amount of $201 for a family of three — one of the lowest grants in the country — is often not a 
strong enough incentive to motivate individuals who need help to comply with program re- 
quirements and “cure” their sanctions. 6 

In the late 1990s, South Carolina conducted research to understand the economic and 
labor market status of individuals who had left TANF. State policymakers were particularly 
concerned about those who had left because of time limits or sanctioning. The South Carolina 
Department of Social Services (DSS) expected that many current and former TANF clients who 
did become employed would lose their first jobs, as well as subsequent jobs, as they dealt with 
barriers to work and started to develop “labor force attachment.” 7 DSS expected that, whether 
working or not, most of these long-term TANF leavers were not doing well economically. 

As anticipated, it was found that — like TANF leavers across the nation — some leav- 
ers in South Carolina were not working; many were working but not steadily; and others were 
stuck in low-wage jobs. Three years after leaving TANF between October 1998 and March 
1999, only 55 percent were employed. Of those who were employed and still not receiving cash 
assistance after three years, approximately 60 percent earned $1,250 or less a month — just un- 
der the 2002 federal poverty level of $15,020 per year (or approximately $1,252 per month) for 
a family of three. 8 Earnings varied, however, depending on the reason for leaving TANF. A 
substantial proportion — about one-third — of those who had left because of sanctions or time 
limits had monthly earnings of only $750 or less. Of those who were unemployed and still not 
receiving cash assistance after three years, only about half (55 percent) had said that they had 
worked at some point during the past 12 months. 9 

Based on these findings, DSS decided to reach out to former TANF recipients and de- 
velop an ERA program to help them succeed in the labor market. The Lewin Group and MDRC 
provided technical assistance to the state as it developed its program plans. Because of DSS’s 
interest in targeting all TANF leavers, the program had multiple goals: to provide services to 
people who were not working, in order to help them obtain jobs, and to provide services to peo- 
ple who were working, in order to help them sustain work and move up in the labor market. 



4 South Carolina Department of Social Services, Office of Family Assistance, 2000-2001. 

5 Goldberg and Schott, 2000. 

6 Edelhoch, Liu, and Martin, 2000. The state’s TANF grant for a family of three increased to $241 in Octo- 
ber 2004, after the period covered in this repoit. 

7 Edelhoch, Liu, and Martin, 2000. 

8 Office of the Federal Register, National Archives and Records Administration, 2002. 

9 Richardson, Shoenfeld, LaFever, and Jackson, 2002. 
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The South Carolina ERA Model 

South Carolina’s ERA program, Moving Up, was a mixed-goals program, providing both 
pre- and postemployment services. The program targeted former TANF recipients who had 
stopped receiving cash assistance between October 1997 and December 2000 for any reason and 
who had never returned to TANF. 10 Although other programs in the national ERA evaluation have 
also targeted TANF leavers (see Appendix Table A.l), Moving Up was the only program that 
focused on long-term leavers. From the pool of eligible TANF leavers in South Carolina, indi- 
viduals were assigned at random to either the ERA group or the control group. (The random as- 
signment process is described further below.) Those who were assigned to the ERA group were 
contacted about participating in Moving Up, and they did so on a voluntary basis. 

The key feature of Moving Up was individualized, one-on-one case management ser- 
vices provided by a career consultant. Feaming from postemployment case management 
evaluations like PESD, Moving Up did not provide a uniform level of services to all participants 
but, instead, attempted to target services based on an individual’ s specific needs. In addition, the 
program’s mixed-goal approach of providing both pre- and postemployment services enabled 
career consultants to work with all participants and to form relationships immediately, rather 
than waiting to engage individuals after they found jobs, as PESD case managers did. 11 Career 
consultants provided services themselves and also referred participants to other providers. 

Depending on participants’ needs, program activities could include counseling on ca- 
reer goals and workforce readiness, job search assistance, short-term education or training, child 
care and transportation assistance, or mental health and other support services. Because partici- 
pation in these activities was voluntary, Moving Up offered modest incentives to keep partici- 
pants engaged in the program; cash rewards or gift certificates were given for such benchmarks 
as finding a job, holding a job, getting a promotion, or completing an education or training ac- 
tivity. (Chapter 2 provides more detail about the program.) 

Characteristics of the South Carolina ERA Site and Its External 
Environment 

Moving Up operated in the six predominantly rural counties that make up South Carolina's 
Pee Dee Region: Chesterfield, Darlington, Dillon, Florence, Marion, and Marlboro. The state chose 
this region for the ERA program because it is the most economically disadvantaged area in the state 
and because the DSS county directors there had experience collaborating on prior efforts. 



l0 At intake, ERA group members’ income levels were assessed, and very few had income above 250 per- 
cent of the federal poverty threshold that was originally set as a criterion for selecting sample members. 
u Rangarajan and Novak, 1999. 



4 




As illustrated in Table 1.1, the populations in the six counties are relatively small. With 
about 128,000 inhabitants in 2003, Florence County had a significantly larger population than the 
others. Unlike its more rural counterparts, Florence County is not as isolated geographically, and it 
has a small metropolitan center that has benefited from job increases in the health service industry. 
Even there, however, nearly half the population live in rural areas. Much like the other counties, 
Florence County is large, and its population is spread out. As a result, it and the neighboring coun- 
ties are subject to many of the problems that prevent the development of a stronger local econ- 
omy, such as geographical isolation and the lack of or inadequate public transportation. 



The Employment Retention and Advancement Project 
Table 1.1 



Labor Force Characteristics of the Pee Dee Region, by County 

South Carolina 



County/State 


Population 

Size 


Labor Force 
Size 


Unemployment 
Rate (%) 


Per Capita 
Income ($) 


2001 


2003 


2001 


2003 


2001 


2003 


2001 


Chesterfield County 


43,136 


43,251 


20,432 


22,469 


7.8 


10.4 


19,972 


Darlington County 


67,656 


67,956 


30,086 


33,847 


6.5 


8.4 


21,880 


Dillon County 


30,907 


31,027 


13,380 


14,555 


10.8 


11.0 


18,033 


Florence County 


126,310 


128,335 


62,592 


72,647 


5.2 


7.2 


25,742 


Marion County 


35,220 


35,113 


14,619 


15,901 


15.4 


15.9 


18,287 


Marlboro County 


28,707 


28,411 


11,458 


13,517 


12.1 


16.9 


17.418 


South Carolina 


4,062,125 


4,147,152 


2,015,600 


2,002,520 


4.8 


6.8 


24.840 



SOURCE: South Carolina Employment Security Commission, 2004. 
NOTE: Unemployment rates are unadjusted. 



When Moving Up began in 2001, the United States was in an economic recession. Dur- 
ing this time, two of South Carolina’s largest industries — manufacturing and trade (in particu- 
lar, retail trade) — experienced a trend of job losses. 12 The Pee Dee Region had relied heavily 
on the manufacturing industry for jobs, with 20 percent to 40 percent of the population working 



l2 South Carolina Employment Security Commission, 2001. 



5 







for manufacturers. 13 The loss of these jobs had a severe effect on the region’s economy, and, 
throughout the follow-up period for this report, job recovery remained slower than the recovery 
of the national economy. 14 As a result, the Pee Dee Region was not able to fully meet the needs 
of its growing labor force, and unemployment rates increased in all six counties, some of which 
have consistently ranked in the top 10 of South Carolina’s 46 counties for having the highest 
unemployment rates (often led by Marion County) and the lowest per capita income. County- 
specific characteristics are presented in Table 1.1. 

During this same period, jobs that were lost in manufacturing and trade have slowly 
been offset by employment gains across the state in the industries of government, education and 
health, and leisure and hospitality (which is often seasonal). Although the Pee Dee Region has 
seen job development in these growth sectors, many of the largest employers (in order of total 
employment) in food stores/services, paper and allied products, textile mill products, and trans- 
portation equipment continue to lose jobs and are not projected to be growth industries. 15 

According to U.S. Bureau of the Census data for 2000 (presented in Table 1.2), the Pee 
Dee counties have relatively high rates of poverty. In 1999, the county rates ranged from 16 
percent to 24 percent, compared with a state rate of 14 percent and a national rate of 12 percent. 

The South Carolina ERA Target Population 

Table 1.3 shows selected characteristics of ERA program and control group members at 
the point that they entered the study. As noted previously, the sample members left TANF be- 
tween October 1997 and December 2000 and did not return to the rolls prior to entering the 
study. As the table shows, the majority (72 percent) had been off TANF at least two and a half 
years at the point of random assignment, which, for the sample in this report, occurred between 
September 2001 and December 2002. As discussed below, each month, 100 ERA group mem- 
bers and 100 control group members were selected. Thus, sample members had left TANF be- 
tween nine months and just over five years before entering the study. Figure 1.1 illustrates this 
timing. For example, Client B left TANF in December 2000, the last month of the target period, 
and was randomly assigned in September 2001, the first month of random assignment; this in- 
dividual had been off TANF for nine months before entering the study. At the other extreme, 
Client A left TANF in October 1997, the first month of the study’s target period, and did not 
enter the study until December 2002, the last month of random assignment; this individual had 
been off TANF for five years and three months before entering the study. 



l3 South Carolina Budget and Control Board, Office of Research and Statistics, 2002-2005. 
l4 DuPlessis, 2004. 

15 South Carolina Employment Security Commission, 2004. 
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The Employment Retention and Advancement Project 
Table 1.2 



Comparison of Percentage of Population Living Below 
Federal Poverty Level in 1999 

South Carolina 





Individuals Living Below 
Poverty Level (%) 


Children Under 18 Living 
Below Poverty Level (%) 


United States 


12.4 


16.6 


South Carolina 


14.1 


18.8 


Chesterfield County 


20.3 


25.0 


Darlington County 


20.3 


27.0 


Dillon County 


24.2 


33.4 


Florence County 


16.4 


22.7 


Marion County 


23.2 


33.6 


Marlboro County 


21.7 


29.4 



SOURCE: U.S. Department of Commerce, Bureau of the Census, 2000 census data. 



As Table 1.3 shows, sample members left TANF for a variety of reasons. The most 
common reason — accounting for 40 percent of the sample — was that the recipient began to earn 
more than the TANF eligibility threshold. Another 19 percent of the sample left the TANF rolls 
because they did not comply with work or training requirements and were sanctioned, and 15 per- 
cent reached the cash assistance time limit. Almost all the sample members in South Carolina are 
women (not shown in the table) and are black (79 percent), and just under half (45 percent) do not 
have a high school diploma or a General Educational Development (GED) certificate. 16 



16 The proportion of sample members who had a high school diploma or GED certificate was estimated us- 
ing educational attainment data in administrative records. Individuals with 12 or more years of education were 
assumed to have a high school credential. 
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The Employment Retention and Advancement Project 
Table 1.3 

Selected Characteristics of Sample Members 
South Carolina 



Characteristic 


Full Sample 


Average age (years) 


31.8 


Race/ethnicity (%) 




Hispanic 


0.4 


Black, non-Hispanic 


78.5 


White, non-Hispanic 


20.4 


Other 


0.7 


Number of children 3 (%) 




0 


1.1 


1 


27.4 


2 


32.9 


3 or more 


38.7 


Age of youngest child 3 (%) 




2 or under 


18.3 


3 to 5 


33.0 


6 or over 


48.7 


No high school diploma or GED 3 ' b (%) 


44.5 


Employed 0 (%) 




In year before random assignment 


67.0 


In quarter before random assignment 


52.4 


Received TANF for 2 years or more d (%) 


27.7 


Time off welfare prior to random assignment (%) 




Less than 2 1/2 years 


28.2 


2 1/2 years or more 


71.8 


Reason for TANF case closure (%) 




Had earnings above eligibility threshold 


40.4 


Sanctioned 


19.4 


Reached time limit 


15.3 


Moved out of South Carolina 


3.4 


Did not complete application 


6.8 


Other 0 


14.7 


Sample size 


2,864 


(continued) 








Table 1.3 (continued) 



SOURCES: MDRC calculations from UI, TANF, and food stamps administrative records from the State of South 
Carolina and UI data from the State of North Carolina. 

NOTES: a This measure is based on the most recent information available in the administrative records at the time 
of random assignment, but it may not be up to date for some sample members. 

b Information on educational attainment was not available. Sample members who had 12 or more years of 
education, according to the administrative records, were considered to have a high school diploma. 

c This table includes only employment and earnings in jobs covered by the North Carolina and South Carolina 
unemployment insurance (UI) programs. It does not include employment outside North and South Carolina or in 
jobs not covered by UI (for example, "off the books" jobs, some agricultural jobs, and federal goverment jobs). 
d This measure is based on TANF receipt in the nine years before random assignment. 

e This measure includes respondents who cannot be located or are missing and cases that were opened in error. 



It is important to note that, by targeting people who left TANF and did not return for a 
considerable period, the Moving Up program may have chosen a group of leavers who, for the 
most part, were making do without services from the TANF program. As stated above, almost 
three-fourths of the research sample had been off TANF for at least two and a half years when 
they entered the study. There was no way to know upfront how many of these individuals 
would need or want the kinds of services that Moving Up offered. 



About the Evaluation 

The Research Design 

Research Questions 

The ERA evaluation focuses on the implementation of the sites’ programs and their ef- 
fects, or impacts. Key questions addressed in this report include the following: 

• Implementation. How did the six counties in the Pee Dee Region execute 
the ERA program, Moving Up? What services and messages did the program 
provide and emphasize? How did career consultants spend their time? 
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The Employment Retention and Advancement Project 

Figure 1.1 

Examples of Duration Off TANF, Report Sample Members 

South Carolina 




Period when report sample left TANF 



Period when report sample was 
randomly assigned 



• Participation. As a voluntary program, did Moving Up succeed in engaging 
a substantial proportion of individuals in services? What types of services did 
people receive? To what extent did the program increase service levels above 
the levels that would “normally” be received, as represented by the control 
group’ s behavior? 

• Impacts. Wi t hin the follow-up period, did Moving Up increase employment and 
earnings, provide employment stability and wage growth, and improve job char- 
acteristics for the ERA group, relative to the control group? 

The Random Assignment Process 

As noted above, to produce reliable estimates of the effects of Moving Up, the evalua- 
tion used a random assignment research design. Eligible individuals were randomly assigned 
either to the ERA group, whose members were eligible for Moving Up services, or to the con- 
trol group, whose members were not eligible for Moving Up services. 

The random assignment process began in September 2001 and ended in January 2003, 
when the pool of sample members who met the target criteria was exhausted. (The sample that 
is analyzed in this report excludes those who were randomly assigned in January 2003, because 
they did not have a full year of follow-up data when the analyses were conducted.) South Caro- 
lina DSS used a local computer program to randomly select and assign eligible sample mem- 
bers from the pool of individuals who had left TANF between October 1997 and December 
2000 and had not returned to the rolls. Each month, DSS first dropped from the pool anyone 
who had begun to receive TANF benefits in the prior month, and then it randomly selected 100 
cases for the ERA group and another 100 cases for the control group. MDRC worked with DSS 
to ensure that, each month, 10 new ERA group members were assigned for each career consult- 
ant and that 10 corresponding control group members were assigned. 17 (Chapter 2 provides 
more detail on sample intake.) 

The Counterfactual: What Is ERA Being Compared With? 

Individuals who were randomly assigned to the control group — who represent the 
counterfactual for the study — were not contacted or informed about Moving Up and were 
treated as though the program did not exist. While other sites in the national ERA evaluation 



l7 January 2003 was the last month that all six participating counties received 10 ERA group members and 10 
control group members. The counties that had not fully depleted their pool of target sample members continued to 
bring new participants into Moving Up through August 2003, but these individuals are not part of the research 
sample. Some individuals who had returned to the TANF rolls after December 2000 were erroneously selected for 
the sample; those individuals were dropped from both research groups and are not included in the analysis. 
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required control group members to participate in already-existing programs, there were no exist- 
ing programs for TANF leavers in South Carolina, so the control group was not subject to any 
particular program as part of the evaluation. 18 

Like Moving Up participants, members of the control group were eligible for services 
in accordance with the rules of programs offering TANF, food stamps, Medicaid, child care, 
and transitional child care and Medicaid benefits. Either on their own initiative or through refer- 
rals other than by Moving Up staff, the control group members could seek out these services as 
well as nonprogram services that were offered in the community through Workforce Investment 
Act (WIA) One-Stop Centers, technical colleges, adult schools and other education providers, 
and employment and training organizations. 

Data Sources 

The data sources for the analyses presented in the report are described below. 

Baseline Data 

At the point of random assignment, South Carolina DSS used administrative records to 
collect demographic, educational, and TANF assistance data on sample members. This informa- 
tion was used to describe the study population (in Table 1.3) and to identify subgroups whose 
results are analyzed separately. 19 

Administrative Records 

Effects on employment and earnings were computed using automated unemployment 
insurance (UI) wage records data, and effects on public assistance were computed using auto- 
mated TANF and food stamp administrative records. One year of follow-up data were available 
for all sample members when the analyses for this report were conducted. 

Program Participation and Implementation Data 

The Employment Retention and Advancement Client Tracking System (ERACTS) — 
developed by South Carolina DSS — provided information on program operations and partici- 
pation, such as the quantity and location of contacts between program staff and participants. 



18 While developing the ERA program. South Carolina DSS was also considering implementing a post- 
TANF program for all prior recipients. This program was not implemented, however, because of limited state 
ftmds, the possible “contamination” of the control group in the study, and the desire to learn first whether the 
more intensive ERA program offered a positive return on investment. 

l9 Baseline data are more limited for South Carolina than for other ERA sites because they were collected 
from administrative records rather than from a baseline survey or a form designed for the study. 
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DSS also provided to MDRC information on incentive payments to Moving Up participants. 
MDRC conducted a “time study” of Moving Up staff, which tracked their activities. Finally, 
information on program operations was available from interviews with Moving Up staff and 
from reviews of participants’ case files. 

The ERA 12-Month Survey 

Information about sample members' participation in program services and about their 
employment, income, and other outcomes was gathered by the ERA 12-Month Survey, which 
was administered to a subset of ERA and control group members approximately 12 months af- 
ter random assignment. 

Sample Sizes 

A total of 3,035 people were randomly assigned between September 2001 and January 
2003 and are known as the research sample for South Carolina. As shown in Table 1.4, this re- 
port focuses on people in a subset of the research sample who were randomly assigned through 
December 2002 and for whom one-year of follow-up data were available; this report sample 
comprises 2,864 individuals — 94 percent of the full research sample. Some analyses in the re- 
port rely on an early cohort of 752 sample members who were randomly assigned between Sep- 
tember and December 2001, for whom two years of follow-up data were available. The survey 
sample — those who completed the ERA 12-Month Survey — is a subset of the sample mem- 
bers who were randomly assigned between February and June 2002. These samples are de- 
scribed further in Chapter 4. 



The Employment Retention and Advancement Project 
Table 1.4 

Overview of Evaluation Sample Sizes, by Research Group 
South Carolina 



Research 

Group 


Random Assignment 
Dates 


ERA 

Group 


Control 

Group 


Total 


Percentage of 
Full Evaluation 
Sample 


Report sample 


September 2001 to December 2002 


1,421 


1,443 


2,864 


94.4 


Early cohort 


September to December 2001 


377 


375 


752 


24.8 


Survey sample 


February to June 2002 


299 


295 


594 


19.5 
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Roadmap of the Report 

As mentioned previously, this report focuses on the ERA program’s implementation 
and impact findings in South Carolina. Chapter 2 further describes the Moving Up program and 
its implementation. Chapter 3 provides information regarding impacts on service receipt. Chap- 
ter 4 covers impacts on employment, earnings, job characteristics, and other outcomes. 
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Chapter 2 

The Implementation of the South Carolina ERA Program 



In order to interpret the impacts of South Carolina’s Employment Retention and Ad- 
vancement (ERA) program, Moving Up, it is first important to understand how the program 
operated and how it was different from what the study’s control group experienced. Drawing 
from field research, automated program tracking data, and a time study of program staff, this 
chapter focuses on how Moving Up was implemented. (Chapter 3 discusses participation in 
employment-related services and activities among both the ERA group and the control group.) 

After a brief summary, this chapter describes how Moving Up was put in place and 
what its structure, staffing, and management were like. It then discusses the program’s services, 
how program staff spent their time, and some differences in implementation across the six par- 
ticipating counties in the Pee Dee Region. 



Key Findings 

Outreach and marketing to potential participants were challenging for the Moving Up 
program. The study’s target group included many individuals who had left TANF years before 
and whose contact information in the state’s database was outdated, making it time-consuming, 
if not impossible, to locate them. Then, after staff had located potential participants, they still 
faced the challenge of marketing the program to individuals who were not required to take part 
in it and who may not have wanted its assistance. 

Moving Up case managers (called “career consultants”) contacted, in person or by 
phone, about three-fourths of the ERA group within a year of their entry into the study. Staff 
reported that, among that group, some individuals did not want to participate; they said that they 
were doing fine or that they were not interested in taking part in a program. 1 Just under half of 
the ERA group participated in Moving Up within that year — some intensively, some cursorily. 

Individualized, one-on-one case management was the core of Moving Up. Career con- 
sultants worked with participants to help them prepare for and find a job, to stay in their current 
job, or to move up. They referred some participants to structured activities, such as job search 
classes and short-term vocational training, and the program provided modest financial incen- 
tives to encourage and reward participation and employment. Because of state budget problems 



'This information is from conversations with the program staff, not quantitative data, so the percentage of 
ERA group members who were in this category is not known. 
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between late 2002 and mid-2003, most counties limited or froze spending on some program 
services, and the intensity of the program diminished. 

Based on MDRC’s field research, job placement was the strongest component of the 
program. Delivery of retention and advancement services was strong in some counties but less 
so in others. South Carolina’s Department of Social Services (DSS) and most program staff 
members had a lot of experience helping people prepare for and find jobs, but they had less ex- 
perience working with employed clients. This was true for most sites in the ERA project, but the 
challenge was compounded in South Carolina by the fact that the program targeted a diverse 
group — some working and some not — and offered all three categories of services: placement, 
retention, and advancement. 

Although the program’s design was uniform across the six counties and although the pro- 
gram coordinator encouraged consistent implementation of Moving Up, its services and intensity 
varied somewhat across the counties. These differences are explored in the following chapters. 



The Framework of Moving Up: Structure, Staffing, and 
Management 

Organizational Structure and Program Funding 

As discussed in Chapter 1, South Carolina's ERA program was designed by the state’s 
Department of Social Services (DSS) to address the needs of families who had left Temporary 
Assistance for Needy Families (TANF). Moving Up operated in the six county DSS offices in 
the Pee Dee Region, which also housed other programs, including TANF and the Food Stamp 
Program. The state DSS office allocated TANF funds for ERA and passed them on to the par- 
ticipating counties, each of which had a DSS director who was responsible for the local opera- 
tion of the ERA program. 

When the program was designed, DSS intended to establish a formal linkage between 
the Moving Up program and the local One-Stop Centers in each county. (The Workforce In- 
vestment Act, enacted in 1998, required the establishment of these centers, which provide uni- 
versal access to a wide range of employment services.) In the largest county in the South Caro- 
lina study, Florence, half the Moving Up staff were located in the local One-Stop, where they 
met with program participants. In all the counties, Moving Up staff sometimes referred partici- 
pants to the One-Stop for services, but a more formal linkage was never operationalized. 2 Mov- 



"As discussed in an earlier report (Anderson and Martinson, 2003), in most sites in the ERA evaluation, the 
linkage between the ERA program and the workforce investment system, which includes One-Stop Centers, was 
based on the linkage that was forged for the TANF system. In the Pee Dee Region, as in some other ERA sites, 

(continued) 
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ing Up also had relationships with the local technical schools and other providers, but these ar- 
rangements were not formal or contractual. 

Program funding varied over time. When the South Carolina study began, in autumn 
2001, Moving Up was fully funded. Over time, the state's budget situation worsened, which led 
to funding reductions in many programs, including Moving Up. These reductions temporarily 
affected the Pee Dee counties’ ability to deliver program services, and, as a result, the program’s 
intensity decreased. Specifically, beginning in late 2002, the counties in the study began to run 
out of funds for Moving Up. Because of the fiscal crisis, the state did not allocate new monies 
for the program until summer 2003. Career consultants remained on the job, but most counties 
froze or limited spending on financial incentives, education and training tuition payments or 
reimbursement, transportation assistance, and other services. 

Staffing and Training 

Moving Up services were provided primarily by case managers (career consultants), 
who were employed by DSS. The largest county in the study had four career consultants; the 
next-largest county had two; and the other four counties each had one career consultant. (Two 
of the ten career consultants left their job in 2003 and were replaced within a few months.) 
These staff members provided individualized case management to participants and connected 
them with other services as needed. In most counties, the career consultants worked with 
agencywide workforce consultants, who built relationships with local employers, developed 
jobs, and shared job listings with the career consultants. 

Most of the career consultants had previously worked for DSS in some capacity — 
many in the state’s TANF program — and all had some prior experience in social services. Be- 
fore Moving Up began operating, the staff received training to learn about its goals, compo- 
nents, and procedures. They also attended a two-day session designed to improve their knowl- 
edge of and skills in recmitment and marketing, and they attended training about how to moti- 
vate clients while they are negotiating life changes. After the program had been operating for 
about a year, staff received additional training designed to improve their knowledge of and 
skills in delivering advancement services and engaging employers. 

Management 

Moving Up was managed by a full-time program coordinator, who worked in one of 
the county DSS offices. Working under and with the guidance of a few DSS administrators in 



the linkage between the welfare and workforce systems was limited by a variety of factors, including the two sys- 
tems’ different goals and target populations and the absence of a coordinated decision-making process. 
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the state office and one of the local county DSS directors, he monitored ERA operations in all 
six counties and reviewed staff performance. He also held monthly staff meetings to review 
program operations, discuss new procedures, and share ideas about working with clients. Each 
county’s DSS director was responsible for the local operation of the program, and typically the 
workforce consultant in each office directly supervised the career consultant(s) in that office. 

To facilitate case management and monitoring, South Carolina developed an automated 
client tracking system, called ERACTS (Employment Retention and Advancement Client Track- 
ing System), specifically for Moving Up. Staff recorded information on each participant, includ- 
ing their status in the program, the activities they were involved in, and their employment status. 
The program coordinator regularly used data from ERACTS to monitor the performance of staff 
and to provide feedback to them. He focused on several items, including the number of contacts 
between staff and clients, the number of individuals participating in the program, the number of 
participants who were placed in jobs, and the number who received a raise or increased their work 
hours. The program manager encouraged staff to contact at least 75 percent of their cases at least 
once and to keep at least 35 percent participating in the program at any given time. Generally, staff 
met these goals. No specific goals were set based on employment outcomes. 

Program management developed different statuses to categorize clients and help staff 
prioritize within their caseloads. “Active” clients were participating in the program and were to 
be contacted (either in person or by phone) at least once a month. “Passive” clients were not 
currently participating but were potentially interested in doing so in the future. Staff had to con- 
tact passive clients monthly also, but this contact could be by letter. After three months, some- 
one who remained uninterested in the program was placed into “refused service” status and was 
not contacted regularly. Box 2.1 presents the South Carolina ERA treatment statuses and the 
corresponding degree of required contact. Based on MDRC’s observations, career consultants 
internalized these statuses and followed the recommended contact guidelines. 

The level of day-to-day supervision within the county DSS offices varied; some super- 
visors were strongly involved in the program and with the staff, while others were not as en- 
gaged. Furthermore, although the program coordinator encouraged consistent practices, the im- 
plementation of Moving Up varied by county, as discussed below. 



The South Carolina ERA Program’s Messages and Services 

Overview of Intended Program Flow 

South Carolina’s ERA program targeted a wide range of individuals who had left 
TANF between nine months and just over five years before. About half were working when 
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Box 2.1 



South Carolina ERA Treatment Statuses 

The South Carolina ERA program used different statuses to categorize individuals who were 
assigned to Moving Up. Following are the statuses and the degree of contact that was re- 
quired for each. 

Pending. The career consultant had begun outreach, but the individual had not yet made a 
decision regarding participation in the program. Career consultants were required to move 
individuals from “pending” to another status within 30 days. 

Active. The individual agreed to participate in Moving Up, and a Career Enhancement Plan 
was developed within 10 days. The individual participated in activities outlined in the plan. 
Career consultants were required to contact active participants at least once a month to follow 
up on activities and check on progress. If an individual was not participating in activities after 
two consecutive months, career consultants revised the status to “passive.” 

Passive. The individual agreed to participate in Moving Up. A Career Enhancement Plan 
was developed within 10 days, but, for two consecutive months, the individual was not ac- 
tively participating in activities outlined in the plan. Or the individual did not refuse services 
but was not interested in participating in Moving Up at that time. Often, a plan had not been 
developed for this passive client. Career consultants were required to attempt to contact pas- 
sive clients once a month to explain the program and remind them of available services. If a 
client was still uninterested after three consecutive months, career consultants revised the 
status to “refused service.” 

Refused service. The individual declined to participate in Moving Up, or a passive client did 
not actively participate after three consecutive months. Unless the individual in this status 
adamantly refused services, career consultants maintained contact twice a year - through mail- 
ings of program newsletters and promotional materials. 

Can’t locate. The individual’s current residence or address could not be determined. 

Moved out of service area. The individual lived outside the Pee Dee Region. 

Other. The individual was deceased or incarcerated or did not fit into any of the statuses 
above. 
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they entered the study, and half were not. The program aimed to help all these individuals 
with employment: Moving Up was designed to help non working clients find a job and to help 
working clients retain their jobs and/or advance in the labor market. 

Figure 2. 1 illustrates the typical paths of individuals through Moving Up. After present- 
ing an overview of the paths, the rest of this chapter discusses in detail the services that were 
provided. (Chapter 3 presents quantitative information on the use of employment-related ser- 
vices by the ERA group and the control group.) 

As described in Chapter 1, for the study in South Carolina, each month — using the 
state’s TANF database — 100 TANF leavers were randomly selected to be in the ERA group 
and were assigned to one of the career consultants, who attempted to contact each individual 
and encourage her to participate in Moving Up. (Another 100 individuals were randomly se- 
lected each month to serve as the study’s control group.) If an individual agreed to participate, 
the career consultant would assess her employment situation, goals, and potential barriers and 
would work with her to develop a Career Enhancement Plan. 

Moving Up did not require any specific activities or services; instead, the content of the 
Career Enhancement Plan was developed through conversations between the individual and the 
career consultant, and the plan varied according to the individual's situation and needs. Typi- 
cally, however, the goal for someone who was not working was to find a job, and so the plan 
included one or more job preparation or placement activities. The goal for a participant who was 
working was typically either job retention or advancement in the labor market, and so the plan 
included one or more postemployment activities. Although the program did not have rigid 
guidelines, typically someone who had recently begun a job or who had a history of unstable 
employment would focus on job retention. Someone who had been working steadily would 
typically focus on moving up in that job or on finding a better job somewhere else. 

The core of Moving Up was individualized case management. For participants who 
were seeking a job, this usually consisted of help preparing a resume, one-on-one assistance 
with the job search process, and assessment of potential barriers to employment. For individuals 
who were focused on job retention, career consultants might help solve workplace problems and 
identify and resolve other issues that might threaten job stability, such as transportation or child 
care issues. For participants who were trying to advance, career consultants might strategize 
with the client about how to approach her supervisor for a raise or how to learn about promotion 
opportunities. 

Nonworking participants were sometimes referred to job search classes at the DSS of- 
fice. (These classes were not exclusively for Moving Up participants.) Both nonworking and 
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The Employment Retention and Advancement Project 
Figure 2.1 

Typical Paths of Individuals Through the South Carolina ERA Program 
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working individuals were sometimes referred to short-term education or training. All partici- 
pants were eligible for a variety of support services, including transportation assistance and 
child care assistance. 

Throughout the different phases of the program, modest financial incentives were used 
to encourage and reward desired behaviors. For example, individuals received $10 for attending 
an initial meeting with a career consultant, $50 for completing a job search class, $50 for keep- 
ing a new job for one month, and $50 for moving from a part-time to a full-time job. Box 2.2 
lists the program incentives, which are discussed further below. 

Based on payment data from the program, 47 percent of the ERA group received at 
least one incentive payment within a year after entering the study, and 16 percent received at 
least one payment of $50 or more. 3 Among individuals who received at least one payment, the 
average amount received during the year was $62. Incentives were used in all six counties but 
were more strongly emphasized in some. (As noted above, the state’s budget problems pre- 
vented most counties from paying any incentives between late 2002 and mid-2003.) 

Based on MDRC’s field research, job placement was the strongest component of the 
Moving Up program. Delivery of retention and advancement services was strong in some coun- 
ties but less so in others. Most staff members had experience helping people prepare for and 
find jobs, as did DSS as an agency. As was true for all the sites in the ERA project, however, 
the staff members and the agency in South Carolina had less experience working with em- 
ployed clients. As discussed in a previous report from the ERA evaluation, all sites in the study 
were challenged to develop strong postemployment interventions, to train staff to deliver them, 
and to engage working clients in program services. 4 In South Carolina, these challenges were 
compounded by the fact that Moving Up targeted a diverse group — some working and some 
not — and offered all three categories of services: placement, retention, and advancement. 

Intake, Assessment, and Client Engagement 

This section and the next two sections present more detail on the services and processes 
summarized above. During the intake period for the South Carolina study, each month, 100 in- 
dividuals who had left TANF between October 1997 and December 2000 and who had not re- 
turned to the rolls were randomly selected using the state’s TANF database. Ten of these indi- 
viduals were assigned to each career consultant. (Another 100 individuals were selected each 
month to serve as the study’s control group; they were not invited to participate in Moving Up 



3 The payment data indicate that 50 percent of the ERA group received at least one incentive payment 
through September 2004 — slightly higher than the one-year percentage. 

4 Bloom et al., 2002. 
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Box 2.2 



South Carolina ERA Financial Incentives 



Modest financial incentives or awards were given to Moving Up participants who met particular 
program benchmarks and accomplished goals outlined in their Career Enhancement Plan. Incen- 
tives or awards were in the foim of coupons or cash and were not retroactive (that is, they were 
not provided for the client’s achievements before the plan was developed). Because of funding is- 
sues, most counties in the study did not pay incentives from late 2002 through summer 2003. 

Initial meeting and development of plan with career consultant 

$10 First face-to-face meeting 

$10 Completion Career Enhancement Plan 

Assessments and prescribed treatments 

$10 Completion of special assessments (for example, vocational rehabilitation, mental health ) 

$ 1 0 Initial visit for treatment 

$50 Completion of treatment 

Job readiness training/ activity 

$10 Completion of 1 week of activity (for example, job club, visits to One-Stop Center) 

$50 Completion of training/activity 

Education/training 

$10 Completion of 1 week of activity (for example, adult education, vocational training) 

$50 Completion of short-term training; incentive paid only after participant received certificate; for 
individual courses, no more than three $50 incentives per year 

$150 Completion of long-term training (for example, 1-, 2-, or 4-year degree or certificate; General 
Educational Development [GED] certificate); incentive paid only after participant received cer- 
tificate or degree 

Obtained/maintained employment (minimum of 15 hours per week) 

$50 Kept new job for 1 month 

$50 Kept new job for 3 months 

$50 Maintained employment for 6 months 
$100 Maintained employment for 9 months 

$150 Maintained employment for 12 months (with no more than 2 voluntary job changes) 

Advancement in employment 

If a participant advanced in the current job or moved to a new job, only 1 incentive awarded at any time. 

$50 Moved from part- to full-time work (30 hours or more per week) and maintained it for 30 days 
$50 Obtained a job with higher wages (an increase of at least 8 percent) and maintained it for 30 days 

$50 Obtained a job with benefits and maintained it for 30 days 
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and did not receive its services.) 5 Information about the individuals — including their last 
known address and phone number, demographic information, and TANF history — was 
downloaded into ERACTS. Career consultants attempted to locate the individuals and encour- 
age them to take part in the program. They sent an invitation letter that included a brochure 
about the program, and they typically followed up with phone calls and additional letters. (Ap- 
pendix C presents the invitation letter, the brochure, and the planning form for Moving Up.) 

It was challenging to locate the selected individuals, and staff did extensive outreach to 
get potential participants into the program. As noted earlier, the target group for Moving Up 
included people who had left TANF as early as October 1997 — years before the ERA study 
began. Although some people had continued to have contact with the DSS office through the 
Food Stamp Program or Medicaid, much of the contact information in South Carolina’ s data- 
base was outdated. Staff often sought current contact information from multiple sources, includ- 
ing various other state departments (such as the Department of Motor Vehicles) and family 
members. Some staff even drove to sample members' last known address and spoke with the 
current residents and former neighbors to get information about individuals’ whereabouts. 

After contacting potential participants, staff still faced the challenge of marketing the 
program to individuals who were not required to take part in it and who may not have wanted 
assistance. They marketed Moving Up by emphasizing its individualized nature — services 
were tailored to participants’ needs, to help each succeed in the labor market — and often by 
encouraging people to think about their broader goals and hopes for themselves and their fami- 
lies. To promote participation, the program provided a $10 incentive to each person who had an 
initial meeting with a career consultant. Management emphasized the importance of effective 
outreach and marketing, and all the career consultants received training in marketing the pro- 
gram and engaging clients. 

It is important to focus on the fact that Moving Up was not a mandatory program. As 
mentioned, this created a challenge for career consultants in engaging participants. Notably, it 
also limited the effect that the program could have on the entire eligible population. When a 
program is mandatory, it can affect even people who do not participate in it. For example, 
someone receiving TANF benefits may choose to find a job on her own, rather than participate 
in a mandatory welfare-to-work employment program. In contrast, there is no reason to think 
that nonparticipants in the South Carolina ERA study would be affected, either positively or 
negatively, by the program. The only chance that Moving Up had to change peoples’ outcomes 
was by engaging them in the program. 



5 As mentioned in Chapter 1, some individuals who had returned to the TANF rolls after December 2000 
were erroneously selected for the sample; those individuals were dropped from both research groups and are 
not included in the analysis. 
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At the initial meeting — which often occurred in the potential participant’s home — 
staff assessed the individual’s employment, educational, and family situation; discussed em- 
ployment goals and barriers; and worked with the person to develop a Career Enhancement 
Plan. Moving Up did not use a standard set of assessment tools; nor did it employ skills testing. 
Instead, career consultants assessed people’ s goals, experiences, skills, and barriers more infor- 
mally, through conversation. The content of the plan varied according to the individual’s situa- 
tion and needs. (Appendix C presents the plan’s template.) Individuals received another $10 
after completing the plan. 

According to program tracking data from ERACTS, 74 percent of the ERA group either 
met in person with or spoke on the phone with a Moving Up staff member at least once during 
the year after they entered the study. In other words, staff succeeded in locating and interacting 
with about three-quarters of the target group for the program. The same tracking data show that 
just under half the ERA group (45 percent) were ever in the active status during this one-year 
period, indicating that they were engaged in the program in some way. 6 Therefore, about a 
fourth of the ERA group were never successfully contacted by program staff, and another fourth 
were contacted but never participated in the program. Staff reported that some of the people 
they spoke with said that they were doing fine and did not need or want help. 7 

It is important to note that the level of activity for clients in the active status varied dra- 
matically. For example, some active participants were working and received a monthly check-in 
phone call from their career consultant, whereas other active participants were engaged in full- 
time education or training. To illustrate this point, consider the number of in-person or phone 
contacts between active participants and staff. According to ERACTS data, among participants 
who were ever categorized as active during the year after they entered the study, the number of 
contacts that year ranges from 1 to 52 per person. The average number of contacts for this group 
is 1 1 (4 in person and 7 by phone). About one-third of these active participants had 1 to 6 con- 
tacts; another third had 7 to 12; and another third had 13 contacts or more. This variation in the 
intensity of participation should be kept in mind when evaluating the program’s effects on out- 
comes, such as employment and earnings, presented in Chapter 4. 

Another way to gauge intensity of participation is to consider a composite measure that 
includes the number of contacts and incentive payments. During the year after entering the study, 
29 percent of the ERA group were ever in the active status; had at least four contacts with staff, at 



6 ERACTS data show that 49 percent of the ERA group — slightly higher than the one-year percentage — 
were ever in the active status between the time they entered the study and April 2004. 

7 This information is from conversations with the program staff, not quantitative data, so the percentage of 
ERA group members in this category is not known. 
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least two of which were in person; and received at least one incentive payment. This indicates that 
just under one-third of the ERA group were engaged relatively intensively in Moving Up. 

Job Preparation and Placement Services 

Placement in a job was usually the goal for program participants who were not working. 
The specific services that were provided to help clients reach this goal varied, but they included 
one-on-one job search assistance and help preparing a resume from a career consultant as well as 
job club classes at the DSS office. Although staff tended to explore individuals’ interests and, 
when possible, tried to help them find a job that fit their interests, they usually encouraged them to 
take a job relatively quickly. Some participants were referred to the local One-Stop Center to look 
for jobs or use assessment tools, such as software that helps identify career interests or skills. 

As noted above, workforce consultants in each of the six Pee Dee counties identified 
job openings at local employers. Although they did this primarily for TANF clients, they often 
shared job openings with the Moving Up career consultants, who then passed them on to pro- 
gram participants. Typically, workforce consultants did not develop jobs for specific clients. 
Most career consultants did not develop jobs themselves (but a few did). Similarly, workforce 
consultants had close connections with local employers, but most career consultants did not. 

If a participant did not have a high school diploma or a General Educational Develop- 
ment (GED) certificate, staff sometimes recommended that she attend classes to help prepare 
for the GED exam. Although most staff believed that a GED is useful in the labor market, typi- 
cally they did not strongly emphasize this as a program activity. Some participants were re- 
ferred to short-term vocational training to build their skills before (or while) seeking a job. 

As noted earlier, Moving Up paid modest incentives to participants for various job 
preparation and placement activities. For example, the program paid $50 if a participant com- 
pleted a job search class or held a new job for one month (see Box 2.2). 

The program also provided support services to participants. Public transportation in the 
Pee Dee Region is limited, so transportation assistance was an important component of the pro- 
gram. Many participants received reimbursement for miles driven in their own cars. If no other 
option was available, some career consultants even drove participants to and from job inter- 
views or classes. 

The designers of Moving Up intended that the program would have funds available for 
child care. Because of state budget problems, however, the funds were never allocated. Instead, 
state administrators modified the rules for allocating TANF transitional child care, which is 
provided to parents who leave the welfare rolls. Rather than providing assistance for two calen- 
dar years following exit from TANF, South Carolina now provides care for 24 months — 
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whether consecutive or not — after exit. (This rule change applies to all eligible parents, not just 
those who were in Moving Up.) The state also operates the ABC Child Care Program, which 
provides assistance for parents whose income is below a certain level. 

Despite initial concerns about Moving Up’s lack of child care funds, staff reported that 
few participants raised child care issues. Most parents who entered the program already had 
care arrangements with family or friends, and those arrangements continued. In some cases, 
staff connected parents with transitional dollars for child care. (A few parents had exhausted 
their transitional care, but this was rare.) 

Staff referred some Moving Up participants to mental health counseling, substance 
abuse treatment, or services for victims of domestic violence. They also helped some partici- 
pants get TANF, food stamps, Medicaid, and other work supports. In 2003, the counties in the 
study began holding monthly “support group” meetings for participants. These were typically 
held during the evening and were facilitated by a career consultant. The main purpose of the 
meeting was to allow participants to share their employment-related experiences and their 
knowledge and coping mechanisms. Sometimes the career consultant also provided informa- 
tion, such as a list of local job openings or a strategy for moving up in a job. According to pro- 
gram management, 10 to 12 participants typically attended a support group meeting. 

Employment Retention and Advancement Services 

For participants who were working, the goal of Moving Up was either job retention or 
advancement in the labor market. Generally, the program encouraged clients to remain in a job 
for a while before trying to move up. To foster job retention, career consultants talked with cli- 
ents about workplace problems and held periodic check-ins to allow participants to share any 
work-related concerns that they had. Although, as noted above, most career consultants did not 
have much contact with employers, a minority checked in with employers about participants’ 
job performance (but only if a participant agreed). 

The goal for participants who had worked steadily for several months was typically to 
advance in the labor market. Reflecting the participant's interest, “advancement” could mean 
getting a raise or additional hours per week at the current workplace or moving to a new job 
with higher pay, more hours, better benefits, a more convenient schedule or location, or getting 
a job in a field of interest to the client. Career consultants helped participants strategize about 
such issues as how to move up in the current workplace and when and how to discuss a raise or 
promotion. Some working clients were placed in short-term vocational training to prepare them 
for a higher-paying job or one with a more convenient schedule. In response to local job open- 
ings, training to become a Certified Nursing Assistant (CNA) was common. 
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Staff were often available — either in the office or by phone — beyond the standard 9- 
to-5 workday, in order to help participants who worked full time or who were occupied with 
family or other responsibilities during the day. At least one career consultant gave his cell phone 
number to participants and told them to call him whenever they needed help. 

Moving Up provided modest financial incentives to encourage job retention (see Box 
2.2). For example, the program paid participants $50 if they remained in a new job for a month, 
another $50 after three months, and another $50 after six months. (Participants who were em- 
ployed when they entered the program and who had been working for fewer than 12 months 
could receive these payments when they reached the benchmarks.) 8 Likewise, the program pro- 
vided financial incentives for advancing in the labor market. It paid $50 to participants who in- 
creased their wage by 8 percent or more, moved from a part-time job to a full-time one, or 
moved to a job with benefits. 

As noted above, it generally was more challenging to deliver retention and advancement 
services than job placement services. To illustrate this, consider the required monthly contact that 
career consultants had with active participants. It is possible to speak with someone who is work- 
ing and to ask general questions like “How are things going at work?” and “Has anything hap- 
pened on the job that you’d like to talk about?” In contrast, it is also possible to ask more directed, 
specific questions that are designed to uncover issues that may affect, either positively or nega- 
tively, job retention or advancement. A question such as “Have you eaten lunch with any of your 
coworkers?” attempts to uncover issues about the client's social network on the job. Questions 
like “What have you learned about other positions at your job?” and “What things do your co- 
workers do that you might be interested in doing?” can prompt a conversation about possibilities 
for advancement. Based on MDRC’s observations, this type of specific probing occurred some- 
times but not consistently across the counties. Also, staff reported that it was often more challeng- 
ing to convince working individuals to participate in postemployment services, particularly ad- 
vancement services; many clients already felt too busy juggling work, family, and other responsi- 
bilities or were comfortable in their current job and did not want to move up or switch jobs. 

How ERA Staff Spent Their Time 

MDRC administered a “time study” in all the ERA sites to better understand the prac- 
tices of the program case managers. The study captured detailed information on the nature of 
interactions between ERA staff and clients and on the topics covered in their interactions. It also 



8 For example, consider a new participant who had been employed for eight months when she entered the 
program. If she remained employed, after one month, she would have received the $100 incentive for nine 
months of employment, and then, after another three months, she would have received $150 for twelve months 
of employment. 
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collected information on how ERA case managers typically spent their time each day. In South 
Carolina, the time study was administered over a two-week period in July 2003. 9 During this 
period, all 10 career consultants recorded their activities each day, using a form designed by 
MDRC. This section presents the key findings from the time study. 

As noted earlier, each career consultant’s caseload in Moving Up grew by 10 potential par- 
ticipants each month. When the time study was administered, staff had an average of 73 participants 
in the active status. Roughly half of the 73 were working, and half were not, and, as noted earlier, the 
active status covered a range of participation intensity. This caseload number does not include the 28 
individuals, on average, who were in the passive status or the 7 individuals, on average, who were in 
the pending status. (See Box 2.1 for the definitions of the statuses.) Caseload sizes varied somewhat 
across career consultants, but staff most commonly had between 61 and 80 active participants. Al- 
though staff sometimes reported to MDRC and the program managers that their caseloads seemed 
too large, the caseload sizes were within the range of those in the other ERA programs. 

Figure 2.2 shows that when the time study was administered, the Moving Up career 
consultants spent about one-third (31 percent) of their time working with clients; this was typi- 
cal across the ERA sites. The career consultants spent a bit more time with nonworking clients 
than with working ones (17 percent of their time, compared with 14 percent). They spent 29 
percent of their time on administrative duties, such as paperwork related to financial incentives 
and entering participants’ activities into ERACTS. Again, these numbers were similar across 
the ERA sites. But South Carolina’s staff spent 13 percent of their time traveling to or from 
program activities or meetings with clients or employers — a higher proportion than in the other 
ERA sites. 10 As noted above, Moving Up career consultants sometimes drove to individuals’ 
neighborhoods (or former neighborhoods) to locate them or to conduct home visits, and they 
sometimes drove participants to job interviews or other program activities; these efforts were 
time-consuming, especially in a rural setting. Career consultants divided the rest of their time 
between job development, outreach to clients, staff meetings, and miscellaneous activities. 

As shown in Table 2.1, South Carolina’s career consultants had an average of about 7 
contacts per day with participants, split evenly among working and nonworking clients. (In the 
other sites in the ERA study, the average number of contacts per day ranged from 5 to 8.) The av- 
erage contact in Moving Up lasted about 19 minutes, and the average contact with nonworking 
clients was somewhat longer than for working clients (21 minutes, compared with 16 minutes). 



9 During this month, one of the career consultants did not receive a new group of individuals to recruit for 
the program — the eligible pool in that county had run out. Furthermore, at that point, most of the six counties 
were still limiting their spending on incentives and other services. Thus, the time period studied is not entirely 
representative of the program at full operation. 

10 In the other sites, the average proportion of time spent traveling was 3 percent. 
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The Employment Retention and Advancement Project 
Figure 2.2 

Summary of How ERA Case Managers Typically Spend Their Time 

South Carolina 



Client contact 
(31 % of all time) 



Contact with working 
clients (14%) 



Contact with non- 
working clients (17%) 




Other activities 
(69% of all time ) 



Administrative duties 
(29%) 



Other (8%) ' 

Staff meetings (5%) 

Outreach to clients 
( 6 %) 



Traveling (13%) 



Job development (8%) 



SOURCE: MDRC calculations from the ERA time study. 

As presented in Table 2.2, during the period studied, just over one-third (38 percent) of 
all client contacts in Moving Up occurred in person. Unlike in the other ERA sites, in which 
most in-person contact occurred in the program offices, the majority of these contacts in South 
Carolina took place in participants’ homes. Of the contacts in South Carolina that did not occur 
in person, most were made by phone. Just over one-third (38 percent) of all contacts were initi- 
ated by the client, rather than a staff person. 

Table 2.3 shows the various topics and activities that were covered during career con- 
sultants’ contacts with participants. The most common activity in South Carolina was a general 
check-in, which accounted for a third (34 percent) of all contacts. This proportion is higher than 
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The Employment Retention and Advancement Project 
Table 2.1 

Extent of Contact Between ERA Case Managers and Clients 
South Carolina 





All Case Managers 


Percentage of work time spent in contact with 


Any client 


30.6 


Working clients 


13.6 


Nonworking clients 


17.0 


Average number of client contacts per day per case manager 


Any client 


6.5 


Working clients 


3.3 


Nonworking clients 


3.2 


Average number of minutes per contact with 


Any client 


18.5 


Working clients 


16.0 


Nonworking clients 


20.8 


Number of case managers time-studied 


10 



SOURCE: MDRC calculations from the ERA time study. 



in most of the other ERA sites. 11 As discussed earlier, Moving Up staff commonly checked in 
with clients who were in the active status, often over the phone, to see whether program activi- 
ties or employment were progressing satisfactorily and whether any problems had arisen. The 
relatively high proportion of time spent in South Carolina on general check-ins suggests that 
Moving Up — compared with most other programs in the study — dealt less with specific is- 
sues regarding job placement, retention, and advancement. 

The activities and topics that were emphasized during the contacts varied with the par- 
ticipants’ employment status. For working clients, general check-in was more common than for 
nonworking clients (38 percent of contacts, compared with 29 percent). For working clients, the 
next most common activities/topics were discussing issues related to financial incentives (27 
percent), exploring specific employment and training options (22 percent), and discussing ca- 
reer goals and advancement (17 percent). For nonworking clients, the most common activities 
were exploring specific employment and training options, assisting with reemployment, and 
general check-in (all near 30 percent of contacts). 



"in the other sites, staff spent between 6 percent and 45 percent of their client-contact time on general 
check-ins; the average across all the sites was 25 percent. 
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The Employment Retention and Advancement Project 
Table 2.2 

Description of Contact Between ERA Case Managers and Clients 

South Carolina 





All Case Managers 


Percentage of all client contacts that were: 


In person 


37.6 


Office visit 


6.8 


Home visit 


23.0 


Employer visit 


0.1 


Visit elsewhere 


7.7 


Not in person 


62.4 


Phone contact 


47.7 


Written contact 


12.1 


Other type of contact 


2.6 


Percentage of all client contacts that were initiated by: 


Staff person 


61.3 


Client 


38.3 


Another person 


0.4 


Number of case managers time-studied 


10 



SOURCE: MDRC calculations from the ERA time study. 



As mentioned above, Moving Up’ s designers thought that it was important for staff to 
be available for participants during nonstandard hours, particularly to accommodate working 
individuals. Reflecting this, during the two weeks that were studied, the time study found that 7 
of the 10 career consultants worked some nonstandard hours, most commonly on the weekend 
(not shown in a table). 



Variations in Implementation Across the Counties 

Although the design of Moving Up was uniform across the six participating counties 
and the program coordinator encouraged consistent implementation, the program nonetheless 
varied somewhat from county to county. This section and those in Chapters 3 and 4 that high- 
light county differences do not identify the counties by name but, instead, refer to them as 
“County 1” through “County 6.” As discussed above, the number of Moving Up staff in each 
county was small — four counties had only one career consultant. In this report, the purpose of 
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The Employment Retention and Advancement Project 
Table 2.3 

Topics Covered During Contact Between ERA Case Managers and Clients 

South Carolina 



Case Managers Working with 



Working 


Non working 


All 


Clients 


Clients 


Clients 



Percentage of all client contacts that included the following topics: 2 








Initial client engagement 


13.2 


18.0 


16.4 


Supportive service eligibility and issues 


9.3 


7.5 


8.8 


General check-in 


37.6 


29.1 


33.6 


Screening/assessment 


2.1 


2.9 


2.5 


Address on-the-job issues/problems 


9.5 


2.9 


6.1 


Address personal or family issues 


13.5 


19.4 


16.9 


Explore specific employment and training options 


22.0 


30.3 


26.4 


Discuss career goals and advancement 


17.0 


13.4 


15.0 


Assist with reemployment 


9.8 


29.9 


21.3 


Discuss issues related to financial incentives or stipends 


27.1 


6.7 


17.7 


Schedule/refer for work experience position b 


NA 


NA 


NA 


Enrollment in government assistance and ongoing eligibility issues 


1.3 


2.7 


2.2 


Assistance with the EITC 


0.0 


0.0 


0.0 


Participation/sanction issues 


14.0 


9.5 


11.2 


Schedule/refer for screening/assessment 


0.4 


5.1 


2.6 


Schedule/refer for job search or other employment services 


1.2 


5.7 


3.6 


Schedule/refer for education or training 


3.5 


3.8 


3.8 


Schedule/refer for services to address special or personal issues 


5.1 


6.6 


5.6 


Provide job leads or referrals b 


NA 


NA 


NA 


Number of case managers time-studied 






10 



SOURCE: MDRC calculations from the ERA time study. 

NOTES: NA = not applicable. 

a Percentages total over 100 percent, since more than one topic could be recorded for each client contact. 
b This measure was not included in the time-study instrument used in South Carolina. 
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discussing county differences is to help explain how Moving Up was implemented and to con- 
nect key differences in implementation to differences in impacts, not to highlight the practices 
and performance of specific counties or individual staff members. 

It is not surprising that implementation varied across the six counties. As discussed ear- 
lier, the county DSS offices had some discretion in operating Moving Up. Over the course of the 
study, it became clear that the county directors had varying degrees of commitment to the pro- 
gram, and staff quality and morale varied substantially. During site visits, as mentioned above, 
MDRC noted that the strength of the counties' retention and advancement services varied. 

The time-study results for the six counties provide useful information about how the 
programs differed. Recall that, across the counties, the average career consultant spent 3 1 per- 
cent of her or his time in contact with clients (Table 2.1). Examining this percentage by county 
shows a range of 19 percent to 53 percent. The average number of contacts per day was 7 (6.5), 
but the range across the counties is 5 to 9 contacts. The average percentage of contacts initiated 
by the participant (rather than by the career consultant) was 38 percent (Table 2.2), but the range 
across counties is 8 percent to 73 percent. 

In some counties, Moving Up operated much as it was designed, focusing on both pre- and 
postemployment services. In other counties, however, postemployment services were less empha- 
sized by management and staff. To illustrate this, consider the activities and topics that were empha- 
sized during contacts with clients in each county. For example, the percentage of contacts with 
working participants in each county that involved discussion of career goals and advancement 
ranged from 7 percent to 48 percent, and the percentage that involved addressing on-the-job issues 
ranged from 1 percent to 56 percent. The counties that spent substantial time on these specific reten- 
tion and advancement topics spent less time on general check-ins. One county, in particular — 
County 6 — stands out with a notably high proportion of time spent on the specific employment- 
related topics discussed above and a low proportion of time on general check-ins. 

These differences suggest that the effects of the ERA program might also differ by 
county. Chapter 3 further explores implementation differences across the counties, by consider- 
ing rates of engagement in Moving Up services and activities, and Chapter 4 presents the pro- 
gram’ s effects for each county. 
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Chapter 3 



The Effects of the South Carolina ERA Program 

on Service Receipt 

Chapter 2 describes South Carolina’ s Employment Retention and Advancement (ERA) 
program, Moving Up, and provides some information on the level of contact that the ERA 
group had with the program, using data from its automated tracking system, ERACTS (Em- 
ployment Retention and Advancement Client Tracking System). This chapter provides addi- 
tional information about participation in the program and other similar services, focusing pri- 
marily on differences between the experiences of individuals in the ERA group (the program 
group members) and those in the control group. Examining these differences is central to under- 
standing the impacts of Moving Up on employment and other outcomes presented in Chapter 4. 
As noted in Chapters 1 and 2, the control group members were not able to receive ERA services 
from Moving Up, but they were able to receive other services from programs and agencies in 
the area, including Temporary Assistance for Needy Families (TANF), the Food Stamp Pro- 
gram, and One-Stop Centers. They were also able to engage in education, training, or other em- 
ployment-related activities that were available in the six counties of the Pee Dee Region. 

This chapter relies primarily on data from the ERA 12-Month Survey, which was admin- 
istered to a subset of ERA and control group members in South Carolina about 12 months after 
they entered the study. A total of 594 sample members (the “respondent sample”) are included in 
the survey analysis. Respondents were asked a series of questions about their contact with case 
managers and with similar staff and about their participation in employment-related activities. Af- 
ter presenting findings for the entire respondent (or survey) sample, the chapter briefly discusses 
some differences across the study’s six counties and presents findings for some key subgroups of 
sample members. 



Key Findings 

Compared with control group members, ERA group members were more likely to have 
had contact with a case manager or employment program during the year after they entered the 
study, but the difference is relatively small. ERA group members were also somewhat more 
likely to have received services to foster job retention and advancement in the labor market. The 
two research groups had similar levels of engagement in most employment-related activities, 
but members of the ERA group were somewhat more likely to participate in vocational training 
and in education or training while employed, and they were more likely to receive assistance 
with transportation. These modest differences are not surprising, given the implementation find- 
ings presented in Chapter 2. 
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Results from the survey suggest that sample members’ contact with case managers and 
their engagement in employment-related activities varied across the six counties in the Pee Dee 
Region. These results should be interpreted cautiously, however, since the survey sample size in 
most of the counties is very small. As in Chapter 2, however, one county stands out when com- 
pared with the other five: The survey data suggest that County 6 produced relatively large in- 
creases in the use of retention, advancement, and job preparation services — although the im- 
pact on job preparation just misses statistical significance. The survey results and the implemen- 
tation results presented in Chapter 2 suggest that County 6 implemented a program that most 
closely mirrored the mixed-goal program that Moving Up’s designers intended. 

The survey shows that the program affected service receipt, compared with control group 
levels, to a similar degree for the subgroups of sample members examined in this chapter. ERACTS 
data, however, show that the subgroups had differing levels of engagement in Moving Up. 



The Intensity and Nature of Contacts Between Clients and Staff 

As discussed in Chapter 2, the core of South Carolina’s ERA program was individual- 
ized case management. Case managers in Moving Up were called “career consultants,” and they 
provided one-on-one assistance and referred participants to additional services as needed. A key 
issue, then, in assessing the strength of the program “treatment” in South Carolina is the inten- 
sity and nature of the contacts between Moving Up staff and clients. The ERA 12-Month Sur- 
vey asked a series of questions intended to capture information about contacts between respon- 
dents and career consultants and other staff from employment and social service agencies. 
Given the nebulous nature of “case management,” it was a challenge to design such questions; 
Box 3.1 describes this effort. 

To estimate contacts between sample members and staff of Moving Up and other or- 
ganizations and agencies that help people find or keep jobs, the analysis combined two survey 
questions into one measure. One question asked whether sample members had had contact with 
“programs or organizations that help people find and keep jobs.” The other question was intro- 
duced with a sentence that referred to “agency staff [who] help people find and keep jobs,” but 
it asked whether respondents had had “contact with a case manager or a staff person from an 
employment, welfare, or other agency.” This chapter refers to this measure as contacts with a 
“case manager or employment program.” 

Table 3.1 presents the program’s impacts on contacts with staff during the year follow- 
ing random assignment. The first column of the table presents outcomes for the ERA group; the 
second presents outcomes for the control group; and the third presents the difference, or impact, 
between the two groups. Because random assignment ensures that there were no systematic dif- 
ferences between the ERA group and the control group when sample members entered the 
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Box 3.1 



Measuring Participation in ERA 

In order to interpret the results of a random assignment evaluation, it is critical to understand the “dose” 
of services that each research group receives. In many studies, this is relatively straightforward, because 
the “treatment” is easy to measure (for example, the number of hours of training or the dollar value of 
incentive payments). In contrast, in many of the ERA programs, including South Carolina's, services 
arc delivered mostly in one-on-one interactions, during which staff advise, coach, or counsel partici- 
pants. This type of service is inherently difficult to measure. In addition, to accurately measure a pro- 
gram’s impact on service receipt, it is important to collect data in the same way for both the ERA group 
and the control group. In practice, this means that survey questions cannot refer to the ERA program in 
particular' but, instead, must ask in general about the kinds of services that ERA provided. 

MDRC sought to measure service receipt in three main ways, using the ERA 12-Month Survey. Each 
approach has both strengths and limitations, and each contributes to the overall analysis: 

• First, the survey asked whether respondents participated in “traditional” employment-related ser- 
vices, such as job search workshops and training classes, and how many weeks they participated 
(see Table 3.3). These services are relatively easy to measure, but they are not the heart of most 
ERA programs, including South Carolina’s. 

• Second, the survey asked how frequently respondents had had contact with staff members from 
employment or social service agencies and where those contacts took place (see Table 3.1). These 
questions arc more central to the ERA programs, but it is difficult to determine which types of 
staff the respondents were referring to. For example, contact with a worker who determines food 
stamp eligibility is likely to be quite different from contact with an ERA case manager. Moreover, 
it may be difficult for respondents to recall the number of such contacts over a one-year period. 

• Third, the survey asked whether respondents received assistance in a variety of specific areas, 
some of which — such as “finding a better job while working” — arc central to ERA (see Table 
3.2). These questions are fairly straightforward, but they do not provide any information about the 
amount of service that was received in each area. 
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The Employment Retention and Advancement Project 
Table 3.1 

Year 1 Impacts on Contacts with Program Staff 
South Carolina 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Any contacts with case manager/employment program 


since random assignment 3 (%) 


44.1 


28.9 


15 9 *** 


0.00 


Average number of contacts with staff/case manager 


4.7 


2.9 


1.9 ** 


0.05 


In person 


2.1 


1.1 


11** 


0.01 


By telephone 


2.6 


1.8 


0.8 


0.20 


Talked with staff/case manager in past 4 weeks (%) 


16.3 


10.9 


5.4 * 


0.06 


Ever met with staff/case manager (%) 


33.5 


22.7 


10.8 *** 


0.00 


At home 


17.3 


3.2 


14 i *** 


0.00 


At workplace 


4.2 


0.8 


3.4 ** 


0.01 


At staff/case manager's office 


25.0 


22.1 


3.0 


0.41 


At school/training program 


4.6 


3.2 


1.4 


0.40 


At other places 


3.9 


2.1 


1.8 


0.21 


Staff/case manager talked with respondent's employer (%) 


Never 


93.1 


95.1 


-2.0 


0.31 


Once or twice 


3.3 


3.1 


0.1 


0.92 


More than twice 


2.6 


0.8 


1.8 * 


0.09 


Don't know 


1.0 


1.0 


0.0 


0.96 


Among those employed since random assignment: h 
Staff/case manager talked with respondent's employer (%) 


Never 


91.3 


93.3 


-2.0 


NA 


Once or twice 


4.1 


4.4 


-0.3 


NA 


More than twice 


3.3 


1.1 


2.3 


NA 


Don 't know 


1.4 


1.3 


0.0 


NA 


Sample size (total = 594) 


299 


295 







SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 



NOTES: See Appendix D. 

a This measure includes respondents who said "yes" to D1 or D3. However, the remaining questions regarding 
number and location of contacts were asked only of respondents who said "yes” to D3. Therefore, there are some 
respondents who reported contact but were not asked about the number and location of contacts. (Question D1 : 
"Have you had any experiences with programs or organizations that help people find or keep jobs since your 
random assignment date?" Question D3: "Since your random assignment date, have you had any contact, in-person 
or by phone, with a case manager or a staff person from an employment, welfare or other agency?") 

b Employment is calculated using the ERA 12-Month Survey and includes those who reported employment since 
random assignment. It includes formal employment and "odd jobs." 
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study, any differences in the groups’ outcomes that emerge over time can be attributed to the 
program intervention. Tests of statistical significance were performed on all impacts presented 
in this report, to determine whether the impact can confidently be attributed to the program. An 
impact is considered statistically significant at the 10 percent level if there is less than a 10 per- 
cent chance that the estimated difference could have stemmed from a program that had no real 
effect. Statistical significance is also presented at the 5 percent and the 1 percent levels. Box 3.2 
gives more information about how to read the tables in this report. 

As Table 3.1 shows, 44 percent of the ERA group reported that they had had contact with 
a case manager or employment program since they entered the study, compared with 29 percent 
of the control group. This difference, or impact, of 15 percentage points is statistically significant, 
as indicated by the asterisks. This difference is relatively small. Recall that the control group was 
not able to receive services from Moving Up but was able to receive services from other programs 
and agencies in the area. The survey did not ask respondents who, specifically, they had contact 
with. The respondents in the ERA group who reported having contact with a case manager or em- 
ployment program include people who had contact with Moving Up but likely also include people 
who had contact with other agencies in the community. 1 As discussed in Chapter 2, program staff 
were challenged to locate and then engage individuals in Moving Up, and the relatively small im- 
pact on contact reflects those challenges. This small difference limits the effect that the program 
could have had on such outcomes as employment rates and earnings. 

It is also worth noting that the survey likely undercounts this type of contact for both re- 
search groups. The survey was administered a year after random assignment, and if sample mem- 
bers had contact with Moving Up or another program early in that year or if the contact was not 
very intensive, they might not have remembered it when asked. It is not unreasonable to assume, 
however, that since the survey captured contact that sample members remembered, it captured 
most of the contact that mattered to participants and was likely to affect their outcomes. 

Table 3.1 presents some details about sample members' contact with program staff. The 
ERA group reported having more contacts with program staff or case managers: 4.7 contacts over 
the one-year period, on average, compared with 2.9 contacts, yielding a difference of 1.9. ERA 
group members were also more likely to have spoken with a case manager during the four weeks 
before the survey (16 percent, compared with 1 1 percent). Again, these differences are small. 



'Recall that the ERACTS data indicate that 45 percent of the ERA group were ever classified as active in 
Moving Up during the year after they entered the study. It seems likely that some ERA group members would 
have had contact with staff from other programs or agencies during the follow-up year. It may be surprising, 
then, that the percentage who repotted on the survey that they had contact with any staff or program was not 
substantially higher than 44 percent. Perhaps the ERA group members who took part in Moving Up were the 
same individuals who took part in other programs. Also, as discussed below, people who had limited contact 
with Moving Up might have forgotten about it when responding to the survey. 
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Box 3.2 



How to Read the Tables in This Report 

Most tables in this report use a similar format, illustrated below. The top panel shows a series of partici- 
pation outcomes for the ERA group and the control group. For example, the table shows that about 27 
(26.7) percent of the ERA group members and about 21 (20.7) percent of the control group members 
participated in an education or training activity. 

Because individuals were assigned randomly either to the ERA program or to the control group, the ef- 
fects of the program can be estimated by the difference in outcomes between the two groups. The “Dif- 
ference” column in the table shows the differences between the two research groups’ participation rates 
— that is, the program’s impacts on participation. For example, the impact on participation in an educa- 
tion or training activity can be calculated by subtracting 20.7 from 26.7, yielding 6.0. 

Differences marked with asterisks are “statistically significant,” meaning that it is quite unlikely that the 
differences arose by chance. The number of asterisks indicates whether the impact is statistically sig- 
nificant at the 1 percent, 5 percent, or 10 percent level (the lower the level, the less likely that the impact 
is due to chance). For example, as shown below, the ERA program had a statically significant impact of 
6.0 percentage points at the 10 percent level on participation in education or training. (One asterisk cor- 
responds to the 10 percent level; two asterisks, the 5 percent level; and three asterisks, the 1 percent 
level.) The p-value shows the exact levels of significance. 

The bottom panel shows the participation outcomes among those who participated in each activity in 
the two research groups. Measures shown in italics are considered “nonexperimental” because they in- 
clude only a subset of the full report sample. Because participants in the ERA group may have different 
characteristics than participants in the control group, differences in these outcomes may not be attribut- 
able to the ERA program. Statistical significance tests are not conducted for these measures. 

Impacts on Participation in Job Search, Education, and Training Activities 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Participated in an education/training activity (%) 


26.7 


20.7 


6.0 * 


0.09 


ABE/GED 


11.0 


10.5 


0.5 


0.84 


ESL 


0.8 


0.2 


0.6 


0.32 


College courses 


11.0 


8.9 


2.1 


0.41 


Vocational training 


8.8 


4.3 


4.5 ** 


0.03 


Among those who participated in each type of activity: 
Average number of weeks participating in 
Job search activities 


8.3 


8.7 


-0.4 


NA 


Education/training activities 


14.5 


12.8 


1.7 


NA 


Unpaid work/subsidized employment 


12.1 


14.8 


-2.6 


NA 






Both research groups reported that contact with staff or case managers often occurred in 
the staff member's office. The ERA group, however, also reported that contact with a case 
manager in their home was fairly common. Very few individuals in either research group re- 
ported that a case manager spoke with their employer. 



Impacts on Service Receipt 

Table 3.2 presents information on the areas in which individuals in the ERA group and 
those in the control group received help during the year after random assignment. As shown in 
the middle of the table, 18 percent of the ERA group reported receiving help keeping a job or 
advancing to a better job (“received help with retention/advancement”), compared with 8 per- 
cent of the control group. This increase of 10 percentage points is statistically significant. As 
shown at the bottom of the table, ERA group members were more likely to have received help 
finding a better job while they were working, to have completed a career assessment activity, 
and to have participated in other miscellaneous activities, such as life skills classes. The middle 
of the table also shows that 21 percent of the ERA group reported receiving help preparing for a 
job, compared with 16 percent of the control group, but this difference just misses statistical 
significance at the 10 percent level (p = 0. 1 1). 

Table 3.3 shows the percentage of each group that participated in various employment- 
related activities during the year after entering the study. Just over half the control group mem- 
bers (53 percent) participated in at least one activity: 16 percent participated in a group job 
search activity; 39 percent conducted an individual job search; and 21 percent engaged in an 
education or training activity. Another calculation (not shown) indicates that if individual job 
search is excluded from the tally, the control group’ s overall participation rate drops to 32 per- 
cent. Although the survey question asked about “an individual or independent job search activ- 
ity, in which you look for a job on your own and sometimes report back to an agency staff 
member,” some respondents may have interpreted the question more broadly and answered 
“yes” if they had looked for work on their own. Thus, this lower participation rate probably bet- 
ter estimates the control group’s involvement in more formal job search activities. 

As Table 3.3 indicates, South Carolina’s ERA program increased participation for only 
a few activities and did not increase the overall participation rate (neither including nor exclud- 
ing individual job search). Compared with control group members, ERA group members were 
somewhat more likely to have participated in vocational training and to have participated in an 
employment or education activity while employed. 

As discussed in Chapter 2, Moving Up participants were eligible to receive support ser- 
vices, including help with transportation and child care. As Table 3.2 shows, more individuals 
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The Employment Retention and Advancement Project 
Table 3.2 

Impacts on Areas in Which Respondent Received Help 
South Carolina 



Outcome (%) 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Received help with support services 


22.0 


12.6 


Q ^ *** 


0.00 


Finding or paying for child care 


13.9 


9.7 


4.2 


0.11 


Finding or paying for transportation 


14.2 


6.0 


g 2 *** 


0.00 


Received help with basic needs 


29.8 


29.9 


-0.1 


0.98 


Housing problems 


11.5 


6.0 


5.4 ** 


0.02 


Access to medical treatment 


22.0 


26.2 


-4.3 


0.23 


Financial emergency 


8.0 


6.5 


1.5 


0.49 


Received help with public benefits 


55.8 


57.4 


-1.6 


0.69 


Getting Medicaid 


44.8 


51.2 


-6.4 


0.12 


Getting food stamps 


45.2 


44.7 


0.6 


0.89 


Received help with job preparation 


21.1 


15.9 


5.1 


0.11 


Enrolling in job readiness or training 


11.6 


9.6 


2.0 


0.45 


Looking for a job 


16.4 


12.2 


4.3 


0.13 


Finding clothes, tools, or supplies for work 


7.1 


6.1 


1.0 


0.62 


Received help with retention/advancement 


17.7 


8.2 


Q ^ *** 


0.00 


Finding a better job while working 


12.0 


3.7 


g 3 *** 


0.00 


Other activities while working 3 


6.8 


3.3 


3.6 ** 


0.05 


Career assessment 


7.8 


4.3 


3.5 * 


0.08 


Dealing with problems on the job 
Addressing a personal problem that makes it 


4.6 


2.4 


2.2 


0.16 


hard to keep a job 


4.5 


2.6 


1.9 


0.22 


Among those employed since random assignment: h 


Received help with retention/advancement 


22.5 


11.4 


11.1 


NA 


Finding a better job while working 


15.2 


5.4 


9.8 


NA 


Other activities while working a 


8.8 


4.5 


4.3 


NA 


Career assessment 


10.0 


5.9 


4.1 


NA 


Dealing with problems on the job 
Addressing a personal problem that makes it 


5.8 


3.4 


2.4 


NA 


hard to keep a job 


5.6 


3.6 


2.0 


NA 


Sample size (total = 594) 


299 


295 







SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 



NOTES: See Appendix D. 

a This measure includes other activities such as life skills and child development classes. 
b Employment is calculated using the ERA 12-Month Survey and includes those who reported employment 
since random assignment. It includes formal employment and "odd jobs." 
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The Employment Retention and Advancement Project 
Table 3.3 

Impacts on Participation in Job Search, Education, Training, and Other Activities 

South Carolina 



Outcome 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Ever participated in any activity 3 (%) 


53.7 


53.3 


0.4 


0.92 


Participated in any employment-related activity 11 (%) 


42.9 


45.0 


-2.1 


0.60 


Participated in a job search activity 


41.0 


44.6 


-3.7 


0.36 


Group job search/job club 


17.6 


16.1 


1.5 


0.64 


Individual job search 


36.1 


39.3 


-3.3 


0.41 


Participated in an education/training activity c (%) 


26.7 


20.7 


6.0 * 


0.09 


ABE/GED 


11.0 


10.5 


0.5 


0.84 


ESL 


0.8 


0.2 


0.6 


0.32 


College courses 


11.0 


8.9 


2.1 


0.41 


Vocational training 


8.8 


4.3 


4.5 ** 


0.03 


Participated in unpaid work/subsidized employment (%) 


5.4 


2.3 


3.1 * 


0.05 


Ever participated in an employment or education 


activity while working (%) 


24.4 


16.0 


8.4 ** 


0.01 


Average number of weeks participating in: 


Job search activities 


3.4 


3.9 


-0.5 


0.52 


Education/training activities 


3.9 


2.7 


1.2 


0.12 


Unpaid work/subsidized employment 


0.7 


0.3 


0.3 


0.20 


Among those who participated in each type of activity: 
Average number of weeks participating in 


Job search activities 


8.3 


8.7 


-0.4 


NA 


Education/training activities 


14.5 


12.8 


1.7 


NA 


Unpaid work/subsidized employment 


12.1 


14.8 


-2.6 


NA 


Sample size (total = 594 ) 


299 


295 







SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 



NOTES: See Appendix D. 

a "Any activity" includes employment-related activities, education/training activities, life skills, and other types 
of activities. 

b Employment-related activities include job search activities, unpaid jobs, and on-the-job training. 
c Education/training activities include adult basic education (ABE), General Educational Development (GED), 
and English as a Second Language (ESL) classes. 
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in the ERA group than in the control group reported receiving help with transportation (14 per- 
cent, compared with 6 percent). The program also increased the proportion who reported receiv- 
ing assistance with housing problems. The increase of 4 percentage points in receipt of child 
care assistance just misses statistical significance at the 10 percent level (p = 0.11). As the table 
also shows, ERA group members were no more likely than control group members to receive 
help with public benefits. Table 3.4 shows that the Moving Up program did not increase receipt 
of services to address mental health issues, domestic violence, or substance abuse. 



Contacts and Services Analyzed by County 

Chapter 2 presents some evidence that the six counties in the South Carolina ERA study 
implemented Moving Up differently. To further explore these differences, this section briefly 
summarizes the participation findings for the counties. Figure 3.1 presents each county’s im- 
pacts on three key participation measures from the survey: contact with a case manager or em- 
ployment program, receiving help with job preparation, and receiving help with retention or 
advancement. For most of the counties, the sample sizes for the survey are small, ranging from 
56 to 244. 2 The results, therefore, should be considered only suggestive. Also, some differences 
that may have reached statistical significance if they were based on larger sample sizes may not 
reach significance. 

As Figure 3.1 illustrates, four of the six counties significantly increased contacts with a 
case manager or employment program, and the increases range from 1 1 to 47 percentage points. 
Further statistical tests were conducted to determine whether the differences in impacts for this 
measure among the counties are statistically significant; they are not. 

County 6 substantially increased receipt of retention or advancement services. The ef- 
fect on job preparation just misses statistical significance at the 10 percent level. The differences 
between the counties’ impacts on receiving help with retention or advancement are not statisti- 
cally significant. The survey results and the implementation results presented in Chapter 2 sug- 
gest that County 6 implemented a program that most closely mirrored the mixed-goal program 
that Moving Up’s designers intended. 



Contacts and Services for Selected Subgroups 

Chapter 4 presents some key employment impacts for selected subgroups of sample 
members, and this section briefly discusses the findings about their participation. (Chapter 4 



2 The sample sizes for the six counties are 56, 57, 58, 59, 120, and 244. 
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The Employment Retention and Advancement Project 
Table 3.4 



Impacts on Receipt of Mental Health, Domestic Violence, and 
Substance Abuse Services 

South Carolina 



Outcome (%) 


ERA 

Group 


Control 

Group 


Difference 

(Impact) 


P- Value 


Received mental health services 


15.1 


15.9 


-0.8 


0.79 


Respondent 


5.5 


4.2 


1.3 


0.47 


Family member 


7.3 


9.5 


-2.2 


0.35 


Both respondent and family members 


2.2 


2.1 


0.1 


0.93 


Received domestic violence services 


2.2 


4.5 


-2.3 


0.13 


Respondent 


0.4 


1.6 


-1.2 


0.15 


Family member 


0.4 


1.6 


-1.2 


0.17 


Both respondent and family members 


1.4 


1.3 


0.1 


0.95 


Received substance abuse services 


3.0 


4.1 


-1.1 


0.49 


Respondent 


2.0 


2.0 


-0.1 


0.97 


Family member 


1.0 


1.7 


-0.7 


0.47 


Both respondent and family members 


0.0 


0.3 


-0.3 


0.35 


Sample size (total =594 ) 


299 


295 







SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 



NOTES: See Appendix D. 



explains why these subgroups are examined.) Figure 3.2 presents the impacts for these sub- 
groups on the three survey participation measures discussed above for the six counties. This 
section also presents some ERACTS information about engagement in Moving Up among the 
ERA group members in each of the subgroups. 3 

One set of subgroups was defined on the basis of employment status during the year be- 
fore entering the study. The recently unemployed sample members did not work in the quarter 
before random assignment, but they did work in at least two of the three quarters before that; in 
other words, they had employment experience but had recently become unemployed. The 
mostly unemployed group did not work in the quarter before random assignment and worked in 
one or none of the three quarters before that. And the recently employed sample members 
worked in the quarter before they entered the study. 

^Findings from ERACTS are not presented by county because the data are too dependent on the diligence 
of individual staff members in entering data into the system. 
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The Employment Retention and Advancement Project 
Figure 3.1 

Impacts on Program Participation, by County 
South Carolina 
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SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 

NOTES: See Appendix D. 

Sample sizes range from 56 to 244. 

The differences in impacts on receiving help with retention or advancement across the counties are 
statistically significant. The differences in impacts on the other two measures across the counties are not 
statistically significant. 
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The Employment Retention and Advancement Project 
Figure 3.2 

Impacts on Program Participation for Key Subgroups 
South Carolina 
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SOURCE: MDRC calculations from responses to the ERA 12-Month Survey. 

NOTES: See Appendix D. 

None of the differences in impacts across the subgroups are statistically significant. 
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The first three sets of bars in Figure 3.2 show participation impacts for these three em- 
ployment-based subgroups. The sample size for the recently unemployed subgroup is small — 
only 77 — so the results should be interpreted with caution. According to the survey, the pro- 
gram did not increase the proportion of this subgroup who had contact with a case manager or 
employment program, received help with job preparation, or received help with retention or 
advancement (the decreases are not statistically significant). The program did increase participa- 
tion for the recently employed and the mostly unemployed subgroups. The differences between 
the subgroups’ impacts on each measure, however, are not statistically significant. The control 
group members who were recently unemployed reported higher rates of contact and service par- 
ticipation than control group members in the other two subgroups, which provided a higher 
threshold for the program to exceed. 

Figure 3.3 shows two measures of engagement in Moving Up for the ERA group mem- 
bers in each of the employment-based subgroups, drawn from the program’s ERACTS data- 
base. The white bars show the proportion of the subgroup’s ERA group members who were 
ever classified as active during the year after random assignment. The black bars show the pro- 
portion who participated more intensively during that year: those who were ever defined as ac- 
tive; had four or more contacts with program staff, at least two of which were in person; and 
received at least one incentive payment. As the first set of bars in the figure shows, the propor- 
tion of ERA group members who were engaged in the program was highest in the recently un- 
employed group and lowest in the mostly unemployed group. (These differences are statistically 
significant.) This is not surprising, since many of the recently unemployed probably were look- 
ing for work when they entered the study, and they welcomed the program’s assistance. 

The second set of subgroups that is examined in Chapter 4 is defined by the length of 
time between the sample member' s exit from TANF and entry into the study. Recent leavers are 
defined as those who left TANF less than two and a half years before they were randomly as- 
signed, and the other group (the “not recent leavers”) left two and a half years or longer before 
entering the study. As Figure 3.2 shows, the program increased the proportion in both sub- 
groups who had contact with a case manager or employment program and who received reten- 
tion or advancement services. Figure 3.3 shows that a somewhat higher proportion of the recent 
leavers were engaged in Moving Up, compared with those who were not recent leavers. (The 
differences between the two subgroups are statistically significant.) This may reflect that indi- 
viduals who had been off welfare longer — without returning to the rolls — had been making 
do on their own and did not need or want the program’s help. 

The third set of subgroups is defined by the reason that the sample member left TANF. 
One group left because they reached the state’s TANF time limit or were sanctioned, and the 
other group left for other reasons (earning too much money, moving out of the state, and so on). 
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The Employment Retention and Advancement Project 
Figure 3.3 

Engagement in ERA Program for Key Subgroups 
South Carolina 
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SOURCE: MDRC calculations from automated Employment Retention and Advancement Client Tracking System 
(ERACTS). 

NOTES: The differences in rates of engagement across the employment-defined subgroups and the subgroups 
defined by time off TANF are statistically significant. The differences in the proportion ever active between the 
subgroups defined by reason for exit from TANF are statistically significant. The differences across the 
subgroups in the proportion more intensively engaged are not statistically significant. 
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Figure 3.2 shows that the program increased the proportion of sample members in both 
subgroups who had contact with a case manager or employment program and received help 
with retention or advancement. (The differences between the two subgroups’ impacts are not 
statistically significant.) Figure 3.3 shows that a slightly higher proportion of the sample mem- 
bers who left TANF because of the time limit or a sanction were active in Moving Up. (This 
difference is statistically significant. The difference in the two subgroups’ rates of more inten- 
sive engagement — indicated by the black bars — is not statistically significant.) 



50 




Chapter 4 



The Effects of the South Carolina ERA Program 
on Employment, Public Assistance, and Income 

South Carolina's Employment Retention and Advancement (ERA) program was called 
Moving Up, and Chapter 3 presents its effects on service receipt; the program modestly in- 
creased participation in short-term vocational training and in job retention and career advance- 
ment services. This chapter presents the program’ s effects on finding jobs, employment reten- 
tion, and advancement in the labor market. It also examines additional outcomes of Moving Up, 
such as participants’ household composition and health insurance coverage. 

The analyses presented in this chapter use administrative records data to compare the 
employment, earnings, public assistance receipt, and income of the ERA group members who 
were eligible for and recruited for Moving Up’s services and of the control group members, 
who did not receive the program’s services. Using data from the ERA 12-Month Survey, this 
chapter also examines whether Moving Up increased the percentage of sample members who 
found jobs that had better hours, wages, or benefits. Most findings in the chapter cover the first 
year after sample members entered the study. Results are presented for the full report sample, 
for an early cohort of sample members, for some key subgroups, and for each of the six counties 
in the Pee Dee Region that participated in the study. 



Key Findings 

• South Carolina’s ERA program had little effect on employment, job 
characteristics, wage growth, and employment retention during Year 1 
for the full report sample. 

Results from the first year of follow-up indicate that Moving Up did not have an effect 
on employment or earnings for the full report sample. Furthermore, the program did not have an 
effect on job characteristics or employment retention. The program did have a positive effect on 
wage growth. This increase, however, affected only a small proportion of the sample and thus 
did not increase average earnings. Employment and earnings impacts were also examined for an 
early cohort, for which two years of follow-up data were available. For this cohort, the program 
did not have any effects in Year 2, and the trends do not suggest that impacts will emerge after 
the two-year period. 

• South Carolina’s ERA program had positive effects on employment and 
retention for three subgroups of sample members. 
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Moving Up produced a large increase in employment and employment stability among 
individuals who had recently become unemployed prior to random assignment, increasing their 
average quarterly employment rate by 15 percentage points. The program also increased aver- 
age annual earnings by about $1,800 and increased the proportion of ERA group members who 
earned more than $10,000 — by 1 1 percentage points above the control group’s average of 1 1 
percent. The program produced a moderate increase in employment among sample members 
who had recently left the Temporary Assistance for Needy Families (TANF) program and 
among those who had left TANF as a result of time limits or sanctions. 

• The program’s effects varied across the six counties that were involved 
in the evaluation. 

Moving Up significantly increased employment in one county in the Pee Dee Region, but it 
did not produce positive effects on employment, earnings, or income in the other five counties. 



The Expected Effects of South Carolina’s ERA Program 

A mixed-goal program like Moving Up — which provides both preemployment and 
postemployment services — is expected to increase employment, employment stability, and 
earnings. By design, the program should increase employment by helping to find jobs for indi- 
viduals who would not have found them on their own and by helping those who lose their jobs 
to find new ones. 

By design, the program should also help ERA group members advance in the labor 
market by helping them to find better jobs or to advance in their current positions. Over time, as 
ERA group members become employed, move to better jobs, and increase their wages in their 
current jobs, their earnings are expected to increase to levels higher than those of the control 
group. Advancement outcomes, such as moving to a better job or obtaining a raise in a current 
job, can take months to achieve. Therefore, impacts on advancement should increase gradually 
but may not fully appear during the one-year follow-up period for this report. 

A number of factors could have reduced the program’s impacts in South Carolina. As 
discussed in Chapters 2 and 3, Moving Up was a voluntary program that faced the difficult task 
of engaging former TANF recipients, including many who had been off TANF for several 
years. Although the program’ s staff located and interacted with a large proportion of the ERA 
group, many of these sample members chose not to participate in Moving Up, which may have 
diminished program impacts. Second, as noted in Chapter 3, some control group members re- 
ceived services that were similar in nature to the services received by the ERA group. Although 
the control group’s service receipt rates were not very high, Moving Up’s impacts on participa- 
tion and service receipt were smaller than expected. Finally, program implementation and par- 
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ticipation impacts varied across the six counties, which may have affected the program’s effects 
on employment and other outcomes. 

The economy can also play an important role in the size of a program’s impacts. If the 
opportunities for advancement in the labor market are limited, it may be harder for the program 
to make an impact on the lives of the people it serves. As discussed in Chapter 1 , during the fol- 
low-up period for this study, South Carolina experienced a decline in the number of manufac- 
turing jobs and a large number of layoffs in that sector. As a result, many of the jobs that were 
available for the population targeted for Moving Up were service jobs, which often pay lower 
wages and provide fewer benefits. Another factor that may have affected the impacts of Moving 
Up is the high employment rates for the control group members: Almost 70 percent of them 
were employed at some point during the one-year follow-up period. The loss of manufacturing 
jobs may have created a hurdle for the program to overcome in affecting advancement, while 
the high employment rates for the control group created an even higher hurdle in the effort to 
increase the rates for the ERA group. 



Data Sources and Samples 

Unemployment insurance (UI) wage data and public assistance payment records are the 
primary data sources for tracking employment, earnings, TANF, and food stamp receipt and for 
estimating impacts on these outcomes. Administrative records data are available for a total of 
2,864 sample members (1,421 in the ERA group and 1,443 in the control group), randomly as- 
signed from September 2001 through December 2002. 1 Monthly public assistance records are 
available for two years prior to random assignment and for one year after random assignment. 
Quarterly employment records are available from South Carolina for three years prior to ran- 
dom assignment and for one year after random assignment. Because the evaluation counties in 
South Carolina’s Pee Dee Region are so close to North Carolina, UI wage data from the State of 
North Carolina were also obtained. 2 

The UI wage data are a good source for producing employment and earning impacts. For 
example, they do not suffer from individual sample members' recall bias. However, they do not 



'Because there is a lag in employers’ repotting to their state UI programs, earnings data obtained by 
MDRC from South Carolina in mid-2004 for this analysis covered the period through the fourth quarter of 
2003. In order to analyze one-year results, the sample had to be limited to those who were randomly assigned 
through December 2002. 

“Among the full report sample, 20 percent are missing North Carolina’s wage data for the first quarter of 
follow-up, and 7 percent are missing two quarters of follow-up data. These missing data are unlikely to affect 
the findings, since only about 5 percent of sample members worked in North Carolina at any point and the 
missing data are largely for the first quarter of follow-up. 
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